Frequently asked questions
Here is a list of answers to some of our most commonly asked questions
Got a question? We hope you will find the answer below but if you can't, don't despair. Try visiting our Contact Us page where we can provide help with any other questions you might have.
Who are DDW?
Dairy Data Warehouse (DDW) is a spin off from Uniform Agri, a Dutch herd management software company. DDW is an independent company specializing on dairy farm data services for dairy farm consultants such as veterinarians, genetic consultants or feed advisers across the globe. Our headquarters and customer services such as our multilingual helpdesk are based in Assen, Netherlands. While some of our developers are also based in Assen, most of them work from various locations mostly in Europe. We also work closely with many subject matter experts and academic institutions such as the veterinary university of Gent in Belgium.
Our mission: Providing access to high quality dairy dataWe live Dairy Data quality. High quality dairy data means for us unmatched level of reliability and richeness of the dairy data set.
Our Vision: Enable the dairy industry to unleash the power of BIG dairy data. We believe Dairy Data becomes most valuable, if a broad range of continuously updated and different data sources is combined and enriched with big data driven algorithms. We are passionate about the potential of BIG Dairy Data to transform modern dairy farming.
What are the key problems DDW addresses?
Data Access: The access to farm data often necessitates manual data export out of the farm management software and transfer to the vet via USB stick or email. This process is cumbersome, can be time consuming and particularly on larger farm operations, farm data becomes outdated quickly.
Data quality (rubbish in – rubbish out).): Farm management advice based on data relies on high quality, reliable farm data. For the farm consultant it can be difficult to assess the data quality of a farm data set. Advise based on inconsistent farm data can lead to wrong advice and actions on farm.
Data Comparabillity (Apples to apples): The comparison between farms in a similar situation can be a powerful tool to trigger action on farm.The comparability of farm data from farms using different data sources however is only possible using a diligent process of data transformation into a single, consistent and comparable data format.
How does the DDW dairy data platform work?
Once the farmer has given DDW permission to connect his farm to the DDW dairy data platform, DDW transfers the farm data once a day from the herd management software on the farm PC to the DDW cloud based data platform.
The data is received on the DDW data platform, where the farm data is cleaned and converted. This unique and science driven process transforms all collected farm data from different data sources into a single data format stored in a single data base.
Dairy farm consultants with a DDW account can ask their dairy farm clients for access to their herd data. Only after explicit authorisation by the farmer is it possible for the respective farm consultant to view the data on the DDW dairy data platform. The data can be viewed through different reports (milk production, reproduction, udder health). Farm consultants can access the reports via their web browser.
What does ETL stand for?
Data Extraction, Transformation (data conversion into a single, consistent format) and Loading (into a central data base, the Dairy Data Warehouse.
What is data cleaning?
Process of cleaning up data in a database that is incorrect, incomplete, or duplicated.
What are conversions?
Conversions are a set of algorithms that detect the type of farm management system the farm data set originated from and transform it into the DDW format. That makes all farm data inside the dairy data warehouse comparable. The conversion process is highly automated but remains under daily surveillance of a dedicated team. Our conversions are frequent updated to follow the constantly changing data base formats of the many data sources we transfer to the dairy data warehouse. Also previously unknown data input errors of farmers and new scientific insights into dairy data handling are integrated into our formulas, to provide our customers with the highest dairy data quality.
What products do DDW sell?
ETL service: Continuous (once daily in most farms) data-extraction, transformation and loading into DDW
Web based reporting tools:
Data Modules (Marts): Data cube extracted out of the Dairy Data Warehouse containing a specific set of key performance indicators. Currently there are the following data modules available: Milk Production, Reproduction, Udder Health.
Functionality Modules: Additional functionality which sits on top of the purchased data modules. Currently there is the functionality module “Farm comparison” available, which allows the comparison of all enrolled farms of an account side by side using the purchased data modules.
How do I select the right products for me?
Everyone who wants to use DDW needs to purchase the ETL service for each farm, they want to connect to the DDW dairy data platform.
A choice needs to be made whether to use HERDNET, which offers data grids but no graphs or HERDNET Pro, which gives access to the full reporting functionality. HERDNET is a good way to get started without adding any additional cost. Most users, who provide regularly dairy farm consultancy to their farm clients choose HERDNET PRO to visualize data for their customers. This makes it easier for farmers to understand their situation. After all “seeing is believing”!
To actually see data inside the reporting tool (HERDNET or HERDNET PRO) the user needs to purchase one or several data modules (Milk production, Reproduction, Udder Health).
Optionally the user can purchase the functionality module “farm comparison”. People like to start their working day by looking at the farmcomparison dashboard to get a quick overview over the situation of the herds under their care. Many consultants also use it to work with farmer groups, who enjoy comparing each other’s farm performances to discover areas for further improvement.
What is the difference between the DDW dairy data platform and HERDNET?
The DDW dairy data platform is a website, where consultants come to access the dairy data of the farms that have given them permission to access their herd data. The dairy data platform is driven by the back office function. It allows users with administrator rights to add other users and manage their user profiles (for example to change a password). Furthermore users can invite herds to join the Dairy Data Warehouse and ask the data providers for data access to their herd data. In return the back office also supports the complete process of connecting a herd data base on a farm PC to the dairy data warehouse in the cloud.
HERDNET is a data reporting tool sitting on the dairy data platform. It can be used to view data inside the respective data modules that have been bought. Users can also upgrade to the HERDNET PRO version adding the ability to visualize data in powerful graphs.
Consultants may also use other data reporting tools such as customized reports the data modules they have purchased.
How do I purchase a product?
Products can be purchased directly on the DDW dairy data platform. Under the tile “Account” and the tab “Account management” the section “orders” can be accessed. Accounts can upgrade HERDNET to HERDNET PRO as well as purchase or discontinue data modules.
All products purchased are always purchased on account level, meaning for any herd enrolled under that account the selected products for that account will be available and invoiced per herd enrolled.
The initial minimum running period for products purchased and herds enrolled on the DDW dairy data platform is 12 months. After that period products and herds can be discontinued at any point in time prior to the 28th of the running month to become effective on the first of the following month. It is always possible with immediate effect to upgrade from HERDNET to HERDNET PRO. Again the invoicing for the upgraded reporting tool will only be charged from the following full month onwards.
How does the invoicing process work?
Independent of the herd enrolment date, there is no fee for the first month of herd enrolment. In the following month at the 28th of the month DDW counts all animals with at least one calving and calculates the per cow fee. This is added to the fees for the ETL, the reporting tool and the respective modules purchased by the account. This sum charged pro rata as a monthly fee on the first of the following month.
Example: A herd enrolled on the 15th of October will appear on the invoice for the first time on the 1st of December for data services delivered in November.
What is the hierarchy inside the DDW dairy data platform?
Dairy farm consultants (for example a veterinarian) have a DDW account (for example for their veterinary practice. Inside an account, there are users (for example all the veterinarians inside a veterinary practice). One or several users can have administrator rights. Users with administrative rights can create other users, assign administrative rights to them and delete users. Administrators can also enrol herds to the account and purchase products such as data modules or an upgrade to HERDNET PRO.
How do I get started?
Simply apply for an account for the DDW dairy data platform by clicking 'Join Us' at the top of this page. Your account is being created within 24hours after reception of the request. We also create a user access affiliated to this account with administrator rights for the person applying for the account. Log in details are sent to you via email: You are all set to log in, create other users and enrol herds straight away. We regularly offer webinars to guide new users through the platform and help them getting the most out of the DDW dairy data platform.
What is a DDW data mart?
The DDW data marts are subsets of the big dairy data warehouse data base. Due to its smaller size a data mart allows faster execution of your queries compared to working directly on the central data base. Data marts are sold as “modules” on the DDW dairy data platform. Currently there the following modules available: Reproduction, Milkproduction, Udder health.
How can I cancel?
The initial minimum running period for products purchased on the DDW dairy data platform is 12 months. After that period herds and products can be discontinued at any point in time prior to the 28th of the running month to become effective on the first of the following month.
Who owns the data?
The data provider (in most cases this would be the farmer) remains owner of the data. The data owner retains therefore full control of the data. The data provider can ask DDW at any point to discontinue the data extraction of his farm data, by using the 'Contact Us' section at the top of this page. As a consequence all consultants with access rights to the respective data will not be able to view these data any longer. DDW will not invoice for herds, where data access is not possible any longer.
What is meant by anonymized data?
Anonymized data are farm data sets without any data that would allow identification of the data provider: for example address details, identification numbers of animals or names of inseminators. DDW uses anonymized data to offer services such as benchmarking.
How does DDW ensure data safety and data security?
Data safety refers to the protection against data loss, data security to the protection against hacking of the data base and data theft. DDW places all its data in a dedicated primary data center in Germany.
Double firewalls are in place to stop unwanted incoming traffic and use Access Control Lists to control who has access to different data. Internet facing servers are placed on a DMZ (De-Militarized Zone) network which is between the internal network and the Internet. All back up life data are based in a secondary data center for availability reasons in case the primary data center goes down. Dual internet lines exist into both data centers.
All servers are protected with antivirus software as well as Intrusion Detection/Prevention Systems.
How can I be sure no one can see farm data without permission of the data provider?
Any consultant who wants to view farm data needs to send a data access request to the data provider. Only once the data provider has accepted the request can the consultant view the data on the DDW dairy data platform.
What legal contracts are put in place to guarantee data privacy?
Data Upload Terms (DUT): The DUT contains in essence all the consents and approvals that are necessary from the farmer (or any other provider of data, where the data is collected by the DDW upload tool), ie. consent to install the DDW upload tool, transfer herd data, and share herd data with those the farmer consents to, as well as to use when anonymized by DDW.
What is the DDW herd data policy?
The Herd Data Policy (the “Policy”) of Dairy Data Warehouse B.V. (“DDW”) sets out the policy commitment of DDW in relation to how herd data is accessed, gathered and used in the course of the DDW business. The Policy is intended to assist those who provide DDW with herd data (in the below referred to as “data providers”) and users of such herd data, to better understand the aims and intentions of DDW. The aspiration of DDW is that the Policy will clarify to data providers how their herd data will be used and to what extent herd data is made available to permitted users, and then with what limitations. Further, by the Policy DDW wants those that use the herd data provided by DDW to understand the rational behind any constraints DDW may have regarding the availability of herd data.
The principles set out in this Policy complements any legal requirements that may follow from laws or regulations and that applies to the operations of DDW.
Each data provider of DDW has consented to the Data Upload Terms. Whereas the Data Upload Terms contain the legal terms and conditions governing how herd data may be accessed, gathered and used by DDW, this Policy seeks to clarify how DDW collects and uses herd data in its operations. Such collection and use is of course always in compliance with the Data Upload Terms.
In this Policy, “herd data” means any type of data or information referring to the herd of a farmer, such as data regarding the identity of the farm, herd size and herd composition, milk yield, feed and feeding data, and data regarding health and breeding status.
Sources and Collection of Herd Data
DDW’s principal source of herd data is dairy farmers, globally. The herd data made available by famers, may also be complemented with herd data collected from dairy related organizations and authorities.
The collection of herd data is enabled by the DDW software application, which enables the DDW system to make a copy of the herd data available on the farm’s herd management system. DDW will never start collecting herd data from a potential data provider without the express consent from that data provider allowing DDW 1) access to its herd data, and 2) the right to install the DDW software application on the relevant IT-system of the data provider, all in accordance with the Data Upload Terms.
The collection of data by DDW is restricted to herd data (as described above). DDW will thus not access general business information, email correspondence, accounting information that is not directly related to the dairy herd, even if DDW would have technical possibilities to do so. If it would be discovered that DDW has such data in its systems for any reason, that data will be deleted immediately.
A data provider can at any time request (by email to email@example.com) that DDW’s connection to the farm management system of a data provider is terminated and that the collection of herd data is discontinued. We will comply with such request and commit to cut the connection within 48 h from such request.
All herd data is stored electronically on secure systems of DDW. The servers of DDW are currently located in Glinde, Germany, and satisfy customary safety, data security and scalability requirements.
Herd Data Use
DDW will only disclose herd data of a data provider to those expressly permitted by the data provider. If another business partner or other relation of a data provider requests access to the herd data of a data provider, such access will not be granted until the data provider has given its express consent to share its herd data with that business partner; relation. A data provider can at any time request DDW (by email to firstname.lastname@example.org), to discontinue the sharing of herd data to one or more of the recipients that a data provider has permitted herd data access to. Further, DDW reserves the right to at its sole discretion restrict release of herd data to certain recipients, if it would consider the release of such herd data be likely to jeopardise the future supply and collection of herd data.
In addition to the above, DDW may provide herd data statistics based on the herd data supplied by the data provider, eg. average somatic cell counts in a country, or average calving intervals in a specified region. Such herd data statistics may be provided to others than those expressly permitted by those data provider contributing to such herd data statistics, since herd data statistics are shared only in fully anonymized form. The requirements of DDW regarding anonymization go beyond legal requirements. DDW will not consider herd data anonymized – and thus will not qualify as herd data statistics – if it is possible to deduct or reverse engineer from that herd data statistics that certain data refers to or originates from a specific data provider.
DDW holds a copy of the herd data collected from the herd data providers, but does not claim ownership to that data. Any ownership rights, copyright or other intellectual property rights referring to the herd data, remains with the data provider. A data provider can at any time request DDW (by email to email@example.com) to discontinue the use of the herd data of that data provider. If so requested, DDW will discontinue the use of such herd data and that herd data will upon request be deleted.
Legal Obligations of DDW
Under the Code of Business Conduct of DDW, it commits to comply with the laws and regulations that apply to its operations, wherever it operates. In order to comply with this commitment, DDW will comply with any order or request from competent authorities that is given to DDW under applicable laws and regulations. So if DDW is requested by a local or governmental authority to provide herd data that DDW has in its custody, DDW will not supply such herd data to that authority unless it is proven that the authority is legally authorized to request such information and DDW by law or regulation is obliged to supply such information to the authority. If DDW is so legally obliged to provide herd data to the requesting authority, DDW will only disclose the data that it is legally required to and hence, will not go beyond what DDW is legally obliged to share by laws and regulations.
The above of course does not apply to sharing of herd data to an authority, if the sharing of herd data to such authority is expressly permitted by the data provider.
Assignment Rights of DDW
Under the Data Upload Terms, DDW reserves an assignment right to affiliates and other third parties. This right is reserved to cater for any legitimate future interests of DDW to perform corporate restructurings, outsourcings and similar rearrangements to its operations. Such potential assignment will not prejudice the rights of any data provider, in relation to any herd data available to DDW. If such assignments would occur, DDW commits not to disclose or extend user rights in relation to herd data to anyone not expressly authorized by the data provider.
If you have any questions regarding this Policy or the practices of DDW in relation to herd data access, gathering and use, please get in touch using the 'Contact Us' page.
Why should I trust DDW?
Our business is founded on trust. If farmers don’t believe that we will always respect their ownership of the data they provide and take extreme measures to ensure the security and safety of their data, our model collapses. Reason enough for us to stick to our promise.
© Dairy Data Warehouse. All Rights Reserved | Click for sitemap