The AI Company That Knows Everything About You

THE DATA YOU GAVE AWAY 📱

You gave it away freely and enthusiastically and with genuine appreciation for the convenience it provided, and you gave it away every day for years, and by the time the implications of the giving became visible enough to produce concern the archive of what you had already given was so comprehensive and so irrecoverable that the concern arrived too late to protect the thing it was concerned about. Your location history, every place you went and how long you stayed. Your search history, every question you asked and what those questions reveal about your fears, your health concerns, your political interests, your relationship status, your financial situation, and the specific shape of your inner life at three in the morning when you asked questions you would never ask out loud. Your purchase history, the specific pattern of your consumption that reveals your values and your vulnerabilities more accurately than any survey could. Your communication history, the emotional tenor and the relational dynamics of your most private conversations, encoded as data and stored somewhere beyond your access 😱

The artificial intelligence companies building the most powerful AI systems in the world require three things that are not equally available to everyone entering the field: enormous computing resources, exceptional human talent, and data, vast quantities of diverse high-quality data representing the full range of human experience, knowledge, language, and behavior that their systems need to achieve the general capability that distinguishes transformative AI from narrow task-specific automation. Computing resources can be purchased with capital. Talent can be recruited with capital and prestige. But data is not primarily a capital problem. Data is a collection problem, and the collection of data at the scale required for training the most powerful AI systems has depended on the accumulation of human behavioral data through platforms that users engaged with primarily for the direct value they received, without fully understanding that their engagement was simultaneously feeding the data accumulation that would eventually power AI systems whose effects on their lives will be as significant as any technology in history.

THE ARCHITECTURE OF KNOWING 🔬

The data that modern AI systems are trained on is not simply the text and images that users explicitly create and share. It is the behavioral residue of every interaction with digital systems: the pause before clicking that reveals hesitation, the scroll speed that reveals engagement versus disinterest, the correction of a typed word that reveals the initial impulse, the sequence of searches that reveals the development of a thought or the progression of a concern, and hundreds of other micro-behaviors that individually seem insignificant but that collectively constitute a record of human cognition and motivation of extraordinary granularity.

When this behavioral data is combined with the content data, the posts and messages and images and videos, and with the social graph data, the relationships and their characteristics, and with the contextual data, the time, location, device, and environment of each interaction, the resulting picture of an individual is more comprehensive and in some dimensions more accurate than the picture that individual holds of themselves, because human self-understanding is constructed through the motivated interpretation of experience that makes some things visible and others invisible, while the data record captures everything indiscriminately.

The implications of this comprehensive knowing in the hands of AI systems capable of acting on it are significant and still unfolding. Targeted advertising, the commercial application that funded the data collection infrastructure, is the most visible and the most mundane. But the same data that targets advertising enables the manipulation of political behavior that researchers have documented through studies showing that algorithmically selected content can shift voter turnout, political attitudes, and social trust in measurable ways. The same data that personalizes content enables the identification of psychological vulnerabilities that sophisticated actors, whether commercial or political, can exploit with a precision that mass-market manipulation was never capable of achieving 🤔

THE REGULATION THAT HASN'T KEPT PACE ⚖️

The regulatory framework governing AI data collection and use in the United States remains inadequate relative to the scale and sophistication of the practices it is theoretically supposed to govern, shaped by the political influence of the technology industry, the genuine difficulty of regulating technically complex practices that legislators and regulators often do not fully understand, and the economic interests of a sector that generates significant employment and tax revenue and whose voluntary cooperation with regulatory processes depends on regulations remaining manageable for industry rather than effective for consumers.

The European Union's General Data Protection Regulation, which came into force in 2018 and which represented the most ambitious attempt by any major regulatory body to impose meaningful constraints on data collection and processing, has produced genuine changes in how companies handle European users' data but has also revealed the limits of data protection law as a tool for addressing the fundamental economic dynamics that drive data collection, because the GDPR's consent framework, which requires companies to obtain user consent for data processing, has in practice produced consent theater: elaborate consent interfaces designed to make the consent that users were going to refuse into the consent that users end up granting through friction and confusion rather than through genuine informed understanding of what they are agreeing to.

The AI-specific regulation that the EU has pursued through the AI Act and that other jurisdictions are developing represents the next generation of the regulatory response, moving from data collection and processing rules to rules specifically governing the use of AI systems that process data, the transparency required about AI decision-making that affects individuals, and the prohibitions on specific high-risk applications including social scoring and certain forms of behavioral manipulation. Whether these frameworks will prove adequate for governing the AI systems of 2025 and beyond, which are more capable and more pervasive than anything the regulators drafting these frameworks had experienced, remains genuinely uncertain 💡

WHAT COMES NEXT FOR DIGITAL PRIVACY 🔮

The future of digital privacy will be shaped by the tension between two powerful forces moving in opposite directions. On one side, the continued advancement of AI systems that extract increasingly granular and actionable information from increasingly diverse data sources, including the sensor-rich physical environment of smart cities, connected vehicles, biometric wearables, and the internet of things, will make the comprehensive surveillance of human behavior simultaneously more technically feasible and more economically valuable for the interests that have built their business models on data collection. On the other side, growing public awareness of the implications of comprehensive data collection, the development of privacy-preserving technologies including differential privacy and federated learning that can enable AI capability without centralizing sensitive data, and the political momentum building in multiple jurisdictions for more serious regulatory constraints will create countervailing pressure.

The outcome of this tension, which will be determined by political choices as much as by technical developments, will fundamentally shape what kind of society exists in 2030 and beyond: whether individuals retain meaningful autonomy over their information and by extension over the decisions about their lives that are increasingly made by systems processing that information, or whether the informational asymmetry that has developed between the large institutions holding comprehensive behavioral data and the individuals whose behavior that data represents becomes so extreme that the concept of informed consent becomes a legal fiction maintained for regulatory compliance rather than a genuine description of the power relationship between people and the systems governing their lives 💛💻✨