# Knowing your data is knowing your AI: Why data provenance matters > The number of reported AI incidents (autonomous vehicles crashing, facial profiling gone wrong…) is steadily rising. As a consequence, the question of… - URL: https://datafund.io/blog/knowing-your-data-is-knowing-your-ai-why-data-provenance-matters/ - Published: 2024-04-19 - Author: Datafund - Tags: privacy, datafund, ai, data-provenance, latest ![](/assets/blog/knowing-your-data-is-knowing-your-ai-why-data-provenance-matters/01.png) The number of reported AI incidents (autonomous vehicles crashing, facial profiling gone wrong…) is steadily rising. As a consequence, the question of what kind of data and where it was taken from to train a model is quickly gaining in importance. [In our previous blog](https://blog.datafund.net/fusingai-web3-improving-the-world-one-dataset-at-the-time-e0d27bc33836), we already briefly introduced the concept of data provenance. This time, we’ll expand on this topic. ![Source: Stanford University: 2025 AI Index Report](/assets/blog/knowing-your-data-is-knowing-your-ai-why-data-provenance-matters/02.png) *Source: Stanford University: 2025 AI Index Report* [](https://hai.stanford.edu/assets/files/hai_ai_index_report_2025.pdf) ## Not all data sources are created equal The challenge of data provenance is not new. With the exponential growth of online information, many researchers have recognised the importance of knowing the origins of data. As [*Huang et al. (2024)*](https://link.springer.com/chapter/10.1007/978-3-031-54252-7_5) put it, *“ … not all data sources are created equal. Some might be rich repositories of accurate information, while others could be tainted with inaccuracies, biases or even deliberate manipulation.”* As AI gradually takes on a more prominent role in critical functions such as driving, navigation, medical diagnosis, it is becoming increasingly important to understand how its training data is sourced and processed. But we are still far from having established ethical and responsible ways of doing so. The American National Institute for Standards and Technology (NIST) noted that the current data-gathering culture is *“focused more on which datasets are available or accessible, rather than what dataset might be most suitable. As a result, the data used in these processes may not fully represent populations or the phenomena being modelled”.* [*(NIST Special Publication 1270 (2022))*](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.1270.pdf) That’s a problem. Because if *“GenAI models are to be scrutinised, adopted and built upon, the data they are trained on must be beyond reproach. Insecure or flawed data collection jeopardises this, undermining the very essence of scientific inquiry… The societal implications of misinformation, biases or flawed model outputs, driven by compromised data, can be deep and far-reaching.”* (Huang et al. (2024)) ## Unverified data is a problem ![Source: Dall-E](/assets/blog/knowing-your-data-is-knowing-your-ai-why-data-provenance-matters/03.png) *Source: Dall-E* So now that we know that training data with shady origins can have (serious) real world ramifications, let’s look at what some of those are. ### Operational issues One of the most straightforward examples is a reduced workload and better output. 90% of work in training an AI model is preparing the data (cleaning, formatting, looking for gaps in the dataset…). Sources with a provable origin, like public governmental databases, can reduce the prep time and increase the chances of a reliable result. For example, which medical recommendation from an AI would you trust more: - One provably trained on millions of MRI images labelled and curated by qualified radiologists? - One by a model trained on unvetted, internet-scraped images? ### Societal issues [*Werder et al. (2022)*](https://kups.ub.uni-koeln.de/53868/1/TMIS_manuscript_DataProvenance.pdf) give another perspective. They show that*“inaccurate and bias-laden AI recommendations have led to* ***incorrect patient treatment****,* ***exacerbated poverty****,* ***wrongful arrest****, and* ***unjust criminal sentencing****”*. They argue that *“given the opaque nature of many AI-based systems, data provenance is essential for understanding AI-based systems’ recommendations”* as it enhances transparency when explaining the model’s outputs. In her 2021 book, [*The Atlas of AI*](https://en.wikipedia.org/wiki/Atlas_of_AI), Kate Crawford gives an example of how engineers used the NIST’s database of police mugshots to train machine vision. The engineers used the photos without seeking the consent of the individuals, who were often already in a vulnerable position when the photos itself were taken and had no control over their future use. Such instances raise challenging questions: - **Does the data perpetuate systemic biases**, like who the AI system is most likely to flag as a criminal? - Was it **obtained with consent**? - In what kind of **context**? - Has the creator been **compensated** for their data? - Was the data even **used with consent**? ![The framework for proactive approach to data provenance (Source: Werder et al: Establishing Data Provenance for Responsible Artificial Intelligence Systems (2022))](/assets/blog/knowing-your-data-is-knowing-your-ai-why-data-provenance-matters/04.png) *The framework for proactive approach to data provenance (Source: Werder et al: Establishing Data Provenance for Responsible Artificial Intelligence Systems (2022))* ### **Legal issues** Failure to account for a dataset’s provenance can have **legal ramifications**, especially in the EU. The new AI Act demands that AI companies operating in the EU provide information *“describing the training methodologies, techniques, and training datasets used… information about their provenance, scope, and main characteristics”* ([source](https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=OJ:L_202401689)). The AI Act also brings attention to the use of proprietary or questionable data in AI models. It recognises that [synthetic data](https://www.tomsguide.com/ai/ai-image-video/adobe-firefly-used-thousands-of-midjourney-images-in-training-its-ethical-ai-model), in particular, may amplify existing biases and issues in original datasets. By adhering to the EU’s standards for data provenance, organisations can navigate compliance challenges more effectively. It’s not a far fetched idea that more countries could follow a similar path in the future. Another case is the blatant stealing of copyrighted material by big AI companies without offering any compensation or an opting mechanism. A slew of copyright lawsuits and regulatory interventions now has AI companies rethinking their data sourcing strategies, as they face the [possibility of running out of high-quality data by as soon as 2026.](https://www.nytimes.com/2024/04/06/technology/tech-giants-harvest-data-artificial-intelligence.html) These companies are striking multi-million-euro deals to access data behind paywalls and within proprietary silos. Underutilised data, sitting within the confines of individual businesses, have become valuable assets, unlocking new segments of the economy. We’re witnessing the [evolution of a data-driven economy](https://www.reuters.com/technology/inside-big-techs-underground-race-buy-ai-training-data-2024-04-05/), where dormant datasets are becoming the fuel for innovation. Private providers will be able to offer verified data of varying qualities to train and enhance AI models, creating a competitive market for data sourcing. This shift underscores the critical role of data provenance in ensuring the reliability and integrity of AI systems. We have written more about it [**here**](https://blog.datafund.net/data-as-a-financial-asset-a-multi-billion-dollar-opportunity-c7f898fd234e?source=collection_home_page----d698ddcdba33-----1-----------------------------------), [**here**](https://blog.datafund.net/the-missing-asset-why-data-is-rwas-next-frontier-efd1b556db25?source=collection_home_page----d698ddcdba33-----0-----------------------------------), [**here**](https://blog.datafund.net/from-hospitals-to-ai-how-data-tokenisation-will-reshape-industries-4e96fb1723e8?source=collection_home_page----d698ddcdba33-----1-----------------------------------) and [**here**](https://blog.datafund.net/the-road-to-2027-how-regulation-and-pilots-are-paving-the-way-for-the-new-data-economy-e5bc8557c0a9?source=collection_home_page----d698ddcdba33-----0-----------------------------------). ![Source: Stanford University: 2024 AI Index Report](/assets/blog/knowing-your-data-is-knowing-your-ai-why-data-provenance-matters/05.png) *Source: Stanford University: 2024 AI Index Report* [](https://aiindex.stanford.edu/report/) ## Provenance = reliability The numerous examples above create far from an exhaustive list. But they do outline a picture why the quality of training data and its provenance are one of the crucial components if we, as a society, want to rely on AI as a trustworthy tool. And while knowing the source of your data isn’t a panacea, it provides valuable insights into the why the model behaves as it does. For instance, understanding the provenance of NIST’s mugshot database wouldn’t change its problematic nature,. What it does is enable researchers and engineers to evaluate and address issues related to its inconsistencies, errors, appropriateness, or even misuse. ## Datafund can help you verify your data At Datafund, we understand the significance of verified data and its potential to shape the future. That’s why we’ve developed a **Web3-based solution to help you quickly and securely verify the provenance of your data.** Our solution is built on [Swarm’s decentralised storage technology](https://www.ethswarm.org/), offering monetisation opportunities while providing full encryption, privacy, and data ownership out of the box. Your keys, your data. Additionally, all metadata and the chain of custody are immutably recorded on the blockchain, ensuring transparency and enabling you to unlock the value of your data. To learn more how to navigate the evolving landscape of data verification and ownership feel free to get in touch with us. ### [**X**](https://x.com/DataFundProject) ## [Linkedin](https://www.linkedin.com/company/datafund) ## or get in touch via: ## info@datafund.io *Edit: Updated October 2025* --- [Knowing your data is knowing your AI: Why data provenance matters](https://medium.com/datafund/knowing-your-data-is-knowing-your-ai-why-data-provenance-matters-1e3484068cea) was originally published in [Datafund](https://medium.com/datafund) on Medium, where people are continuing the conversation by highlighting and responding to this story. --- *First published on [Medium](https://medium.com/datafund/knowing-your-data-is-knowing-your-ai-why-data-provenance-matters-1e3484068cea) on 2024-04-19.*