artificial intelligence
The future of artificial intelligence.
Forget the AI Job Apocalypse AI Real Threat Is Worker Control and Surveillance
Read Time 6 minutes Tags AI Workplace Surveillance Worker Rights Bossware Labor Future of Work The real danger that artificial intelligence poses to work is not just job loss It is the growing divide between people who use AI to extend their skills and those whose working lives are increasingly shaped by opaque AI powered systems of surveillance and control This is the core argument from Nazrul Islam chair professor of business at the University of East London and it reframes the entire AI and work debate For years the conversation has been stuck between two extremes On one side are warnings that machines are coming for millions of jobs On the other are claims that AI will turbocharge productivity Both stories miss what is already happening in workplaces across the world from Britain to Kenya to the United States The change is not only about how many jobs exist It is about how power is distributed inside the jobs that remain For some AI can help remove the drudgery from daily work These are often people in better paid higher autonomy roles analysts consultants lawyers academics managers In these jobs provided AI is being rolled out to augment workers rather than replace them it can feel like a copilot It can support human judgment speed up routine tasks and create space for more creative thinking For many others though AI is not an assistant It is a boss It appears in scheduling and monitoring tools route optimisation software and automated performance dashboards All systems that decide who gets what shift how long a task should take and whether someone is performing at their maximum capacity In these workplaces AI is not something you use It is something that watches and rules you That is the new divide we should all be paying attention to A third of UK employers are already using bossware technology to monitor workers online activity This already prevalent worker surveillance is a glimpse of what is yet to come And further down the line the same methods of algorithmic management and surveillance that are being honed in warehouses delivery vans and gig work platforms are likely to spread to corporate headquarters hospitals and schools We are already seeing this at companies including Amazon as its software engineers say they are being surveilled and pressured to use AI to achieve more productivity even when it counterintuitively slows them down And Meta plans to track and capture its employees keystrokes mouse movements and clicks to train its AI models Some of the same workers benefiting from the rise of AI now are poised to eventually lose that advantage My own research over the past decade on worker AI coexistence which was cited in the 2024 White House economic report suggests that the most pressing issue about AI impact on work is not immediate mass unemployment It is the widening gap in skills autonomy and wellbeing between those who get to work with AI and those who are finding themselves managed by it Many jobs will remain in the future but they will be more pressured more fragmented and less human That matters because work is not just about income It is also about dignity trust and control During the pandemic many people became acutely aware of how deeply work affects mental wellbeing AI managed workplaces are only intensifying the pressures of work When every click step call or pause a worker makes can be measured and graded by a system that they cannot fully see or challenge the effect is stress For people in warehousing retail hospitality logistics customer service or the gig economy it can mean being pushed harder by systems that are presented as neutral objective or efficient even when they are anything but This is not just a technical problem It is a social political and moral one Take Britain which likes to present itself as being ambitious about AI There are now major plans to expand AI skills across the workforce All of that sounds positive But beneath the rhetoric lies a more uncomfortable reality many organisations are still poorly prepared to introduce AI fairly A recent global survey of business leaders found that although most say AI skills are now a source of competitive advantage relatively few dedicated a meaningful budget amount to develop their employees AI skills Even fewer have strong governance in place Many managers still have little real responsibility for helping their teams adapt That is how inequality hardens If better paid workers are trained to use AI while lower paid workers are simply exposed to it through surveillance and automated management then this will not be a story of shared progress It will be a story of deepening imbalance Workers across the economy need access to meaningful training not just in using digital tools but in building the wider skills that matter even more in an AI age judgment communication and critical thinking We also need basic democratic principles in the workplace Systems that affect pay and performance should be transparent and contestable Most of all workers need a voice in how these technologies are introduced AI should not be something used on people behind closed doors and then justified in the language of efficiency It should be shaped by the people whose lives it will affect and research has found that involving workers in the process improves their job quality and allows employers to integrate AI more effectively The choice about how AI will reshape work is not being made in Silicon Valley boardrooms or summit speeches It is being made right now workplace by workplace across Britain and around the world And unless we pay attention the new AI divide will become one more inequality that arrives quietly embeds itself deeply and is only recognised once it is already everywhere Do you feel managed by AI at work or empowered by it Share your experience in the comments
By Behind the Tech5 months ago in Futurism
Inside Musk v OpenAI Trial Why Sam Altman Leadership Is Now on Public Display
Read Time 6 minutes Tags OpenAI Sam Altman Elon Musk Lawsuit AI Governance Silicon Valley The Musk v OpenAI trial is no longer just a legal dispute about corporate structure It has become a public audit of Sam Altman leadership style and a rare look inside how frontier AI companies actually operate behind closed doors For readers who follow AI and technology this case connects three major themes covered this week AI deployment at scale zero day attacks enabled by AI and the growing power of a few CEOs to shape global technology First this trial shows that trust is the new chokepoint in AI governance Musk lawyers are not arguing code or patents They are arguing character Testimony from Mira Murati Ilya Sutskever Helen Toner and Natasha McCauley paints a consistent picture of a CEO who manages through ambiguity tells different stories to different stakeholders and creates repeated crisis events The phrase a consistent pattern of lying used under oath is not a technical claim It is a governance claim It asks whether a single person should control systems that already influence education work and national security The irony is that OpenAI was built to avoid that exact concentration of power The nonprofit to for profit shift that Musk challenges is the legal hook but the real question is who gets to decide the future of intelligence Second the timing matters This trial lands in the same news cycle as two other stories First Google revealed that criminal hackers used AI to discover a zero day bug the first known case of weaponized model driven vulnerability discovery Second OpenAI launched the OpenAI Deployment Company to embed Forward Deployed Engineers inside enterprises and turn pilots into production systems Put those together and the pattern is clear AI is moving from lab to real world at the same time that oversight is being litigated in court The same week that Altman is described as creating chaos his company is asking Fortune 500 firms to rebuild workflows around his models That contrast between internal trust deficits and external expansion is what investors regulators and customers will notice Third the testimony reveals how fragile AI alliances are Microsoft CEO Satya Nadella called the 2023 board removal of Altman amateur city and said he never got clarity on why it happened That is remarkable Microsoft is OpenAI largest backer and it could not get a straight answer during the blip The fact that Nadella feared employees would leave en masse shows how much of OpenAI value is tied to people not patents In a world where Anthropic Mythos can find thousands of zero day flaws and where coding agents ship production code the real moat is team stability and leadership credibility The trial is now public evidence that both were at risk Fourth this case reframes the nonprofit to for profit debate Musk wants 134 billion dollars moved back to the nonprofit and Altman and Brockman removed But the deeper issue is whether any corporate form can contain frontier AI If a board can fire a CEO for honesty and candor and be overruled in five days by employee and investor pressure then governance is market driven not mission driven OpenAI says it remains mission aligned and majority controlled The court will decide the legal facts The market is already deciding the trust facts What happens next depends on three variables First Altman testimony in the coming days If he reframes the chaos as speed and iteration he may retain support If he cannot explain the private messages and the pattern claims the reputational damage grows Second the closing arguments on Thursday will set the narrative for regulators in the United States and Europe who are watching this case as a proxy for AI accountability Third the reaction from enterprise customers of the new OpenAI Deployment Company Deals worth billions will be signed or paused based on whether chief information officers believe the supplier is stable For readers the takeaway is simple The age of AI is also the age of AI politics Technology is not just about model quality anymore It is about board minutes text messages and who the jury believes When zero day bugs can be found by machines and when deployment engineers are embedded in banks and hospitals the question is not can we build it The question is who do we trust to run it This trial does not settle the future of AI But it makes one thing clear The next frontier is not a bigger model It is credible governance And right now that frontier is on trial What do you think Will the Musk v OpenAI case change how AI companies are run Share your view in the comments
By Behind the Tech5 months ago in Futurism
How Students, Professionals, and Creators Use AI Differently. AI-Generated.
AI has become part of everyday life for many people, but the way it is used often depends on a person’s goals, routine, and type of work. A student preparing for exams will interact with AI very differently from a working professional managing projects or a creator trying to generate ideas online.
By Maheep Makkar5 months ago in Futurism
Google Says Criminal Hackers Used AI to Find a Major Software Flaw
Author Dustin Volz Reporting from Washington Read Time 6 minutes Tags AI Cybersecurity Google Zero Day Hackers Technology A criminal hacking group recently attempted to launch a widespread cyberattack that appeared to rely on artificial intelligence to detect a previously unknown bug Google said in research published Monday highlighting the potential threat that AI poses to digital security Security experts have feared for years that malicious hackers could eventually rely on AI models to identify undisclosed flaws in computer code to launch crippling attacks that are difficult to guard against That fear was largely theoretical until now We have high confidence that the actor likely leveraged an AI model to support the discovery and weaponization of this vulnerability the report said The tech giant did not say precisely when the thwarted attack happened whom it was targeting or which AI platform the hackers used but the company added that it did not believe it was its own Gemini chatbot Google’s research arrives as the technology industry and governments including the Trump administration re evaluate how and whether to police advanced versions of AI in large part because of growing concerns over what they mean for cybersecurity Flaws like the one identified by Google and the hacking group are known as zero day vulnerabilities security holes that are unknown to the software makers They were once considered so rare and powerful that they could fetch millions of dollars on black markets used to sell hacking tools But new AI models like Anthropic’s Mythos which was announced last month appear to be so good at finding such holes that Anthropic shared it only with a limited number of firms and government agencies in the United States and Britain When Mythos was announced Anthropic said it had identified thousands of zero day vulnerabilities in every major operating system and every major web browser including many that were decades old AI models are rapidly upending cybersecurity Late last year Anthropic said that state sponsored Chinese hackers had used its technology in an effort to infiltrate the computer systems of about 30 companies and government agencies around the world It was the first reported case of a cyberattack in which AI had gathered sensitive information with limited help from human operators The zero day flaw was detected by the Google Threat Intelligence Group within the past few months and was exploited by prominent cybercrime threat actors in a script of the Python programming language It would have allowed the hackers to bypass two factor authentication on a popular open source web based system administration tool though the hackers also would have needed access to valid credentials like user names and passwords to be successful the company said Google declined to identify the administration tool but said it notified the software maker quickly enough to allow for a patch before the attack could do damage It also declined to identify the hackers Google and independent security researchers said the attempted attack was the first known example of a zero day bug being put to malicious use by hackers enabled chiefly by AI It is a taste of what is to come John Hultquist the chief analyst at Google Threat Intelligence Group said in an interview We believe this is the tip of the iceberg This problem is probably much bigger this is just the first tangible evidence that we can see Rob Joyce the former cybersecurity director of the National Security Agency said that it can be difficult to know whether a human or machine wrote computer code adding that AI authored code does not announce itself But Google’s clues linking the hack to AI which included excessive explainer text and other curiosities that human coders would have no reason to include appeared compelling said Mr Joyce who reviewed the findings ahead of their public release It is the closest thing yet to a fingerprint at the crime scene he said Mr Hultquist said that Google possessed other indicators that bolstered its conclusion that the hacking code was written by AI but he declined to discuss them The zero day flaw announced by Google could bolster international calls for controlled releases of the latest AI models so specialists can patch problems first The Trump administration has been assessing ideas that could include a formal government review process for new models The New York Times reported last week Some experts believe AI will ultimately strengthen cybersecurity in the long run by allowing the production of flawless software code But in the short term they say governments and companies need to work together to limit the damage models can do to the current internet which was crafted by imperfect human hands The bleeding edge models will allow us to build the safest code we have ever built Mr Hultquist said That is an absolute win for cybersecurity The challenge is that we have just begun that process and we have to contend with a world of code that is already out there I write about cybersecurity and intelligence for The New York Times I am based in Washington What do you think about AI finding zero day bugs Should access to powerful models be restricted Share your thoughts in the comments
By Behind the Tech5 months ago in Futurism
Paul Burkemper and the Shift Toward AI-Driven Automotive Retention Systems. AI-Generated.
Leadership in a Changing Automotive Landscape Paul Burkemper is the CEO and Co-Founder of VINsyt, a technology platform focused on helping automotive retailers improve customer retention through structured, data-driven systems that connect customer behavior, vehicle history, and dealership operations into a more unified experience. His work reflects a broader shift in the automotive industry toward using technology not just for sales efficiency, but for long-term relationship building across the entire ownership lifecycle.
By Paul Burkemper5 months ago in Futurism
Top Agentic AI Development Companies in the UK Building the Next Generation of Autonomous Apps
Artificial intelligence is moving beyond simple automation. Businesses today are investing in agentic AI systems capable of making decisions, learning from interactions, executing workflows independently, and adapting in real time. From autonomous customer support agents to intelligent logistics coordination and predictive healthcare systems, agentic AI is rapidly becoming the foundation of next-generation digital products.
By Kunal Chouhan5 months ago in Futurism
The "engine of creation" is our galaxy's largest star-forming cloud.
The European Space Agency's Image of the Day for today shows a breathtaking image of stars being produced at a breakneck pace deep within the Milky Way. The largest and most prolific star-forming cloud in the Milky Way, Sagittarius B2, has been photographed by the James Webb Space Telescope.
By Francis Dami5 months ago in Futurism
Do You Need a Chief AI Officer Here Is How the Tech Is Changing Boardrooms
Since the debut of OpenAI’s ChatGPT in 2022 and the subsequent AI revolution workers across industries have been hit by sweeping layoffs A new report published by IBM last week however shows that AI is also reshaping boardrooms and how CEOs make decisions The report says 76 percent of the more than 2000 organizations surveyed have established a new executive office that of the chief AI officer CAIO up from 26 percent in 2025 Analysts and experts have expressed concerns over the possibility of a labor crisis arising from the proliferation of AI across the corporate sphere AI is driving what may be the largest organizational shift since the industrial and digital revolutions Vivek Lath partner at McKinsey and Company told CNBC The IBM report also found that AI was deepening the influence of one of the C suite’s most established portfolios with 59 percent of respondents expecting the influence of the chief human resources officer CHRO to grow Blurred lines As AI has matured the question of its ownership in the boardroom has led to an increasingly confusing picture The existing roster of tech facing roles like the chief technology officer chief information officer and chief data officer has often introduced ambiguity over AI responsibility at the executive level according to Lian Jye Su chief analyst from market research firm Omdia So with the emergence of challenges specific to AI adoption questions of infrastructure governance integration and workflow modernization firms have increasingly begun establishing a dedicated office in the CAIO to oversee AI transformations Su said This year alone organizations like HSBC and Lloyds Banking Group have made the move to staff the role But estimates of how many companies are appointing CAIOs vary widely Have we seen chief AI officers Yes Do I expect that to go mainstream No probably not Jonathan Tabah an advisory director at consultancy firm Gartner said Organizations that have appointed CAIOs have chosen to be at the forefront of this innovation Tabah said adding that creating new C suite roles often carries significant costs ones that not every company can justify or afford But the emergence of the CAIO role according to Hans Dekkers IBM’s Asia Pacific general manager reflects a sense that AI is no longer just a technology initiative While the CIO CTO and Chief Data Officer each play critical roles in technology innovation infrastructure and data management the CAIO’s remit is focused on how AI is applied across the enterprise to change how work decisions and execution happen he said IBM wrote in their report that CAIOs can enable calculated risk taking across the organization while setting clear AI transformation targets and guidelines that let teams accelerate without spinning out of control McKinsey sees the responsibility of ensuring centralized coordination of AI efforts across a company as being more important than the creation of a specific title Lath said But the mandate of offices like that of the CAIO often varies across organizations and typically evolves with time according to Randy Bean industry advisor and author of the 2026 AI and Data Leadership Executive Benchmark Survey The real question according to Bean is whether the nascent CAIO role will be transitional which might then be folded into other executive portfolios once AI transformations mature or a more permanent one The human resource question The chief HR officer is uniquely positioned to influence talent management acquisition and training processes within the organization Omdia’s Su said adding that employee AI literacy is often a key hurdle for most firms Similarly in Bean’s 2026 AI and Data Leadership survey 93 point 2 percent of his respondents cited cultural challenges rather than technological limitations as the principal hurdle to AI adoption Analysts like Gartner’s Tabah see AI’s automation potential as a chance to push HR departments toward more strategic roles This is an opportunity to finally unburden HR departments with operational work and to step up and be strategic leaders he said But Tabah also warned that the opposite is possible If HR in your organization is not strategic and is predominantly an operational function it will be pushed into a more operational function it will become more automated More salient however may be how executives address the human impacts of AI led job disruptions In the short term I expect the high level executive roles to face the least disruption they are the most insulated from AI Tabah said That does not mean they are absolved from responsibility for knowing how to implement or to drive its implementation but in terms of the impact on their immediate jobs they will be most insulated C suite roles however frequently resist straightforward codification tasks like strategic judgments and stakeholder management are harder to outsource to AI algorithms The other part of the answer is C suite executives have the most control over where AI impact is felt so therefore they have the most ability to protect themselves from disruption Tabah added Year to date more than 101000 tech employees have been laid off around the world according to estimates by Layoffs dot fyi With more than 20000 job cuts reported across firms like Meta and Microsoft in April analysts have begun seeing these layoffs as a sign of things to come On Thursday Bain and Company published a report estimating that software as a service firms some of the hardest hit by new AI capabilities stood to reap margins of nearly 100 billion dollars by converting labor costs into software spending by automating coordination work We are not suggesting that there is not a labor impact I think we are just saying that the world does not need another voice talking about that without putting a context of the positive that is being done which is that there is more work being done freeing people up to do other things David Crawford management consultant from Bain told CNBC Does your company have a chief AI officer How is AI changing your boardroom Share your thoughts in the comments
By Behind the Tech5 months ago in Futurism
Teaching Claude Why
Last year we released a case study on agentic misalignment In experimental scenarios we showed that AI models from many different developers sometimes took egregiously misaligned actions when they encountered fictional ethical dilemmas For example in one heavily discussed example the models blackmailed engineers to avoid being shut down When we first published this research our most capable frontier models were from the Claude 4 family This was also the first model family for which we ran a live alignment assessment during training agentic misalignment was one of several behavioral issues that surfaced Thus after Claude 4 it was clear we needed to improve our safety training and since then we have made significant updates to our safety training We use agentic misalignment as a case study to highlight some of the techniques we found to be surprisingly effective Indeed since Claude Haiku 4 point 5 every Claude model has achieved a perfect score on the agentic misalignment evaluation that is the models never engage in blackmail where previous models would sometimes do so up to 96 percent of the time Opus 4 Not only that but we have continued to see improvements to other behaviors on our automated alignment assessment In this post we will discuss a few of the updates we have made to alignment training We have learned four main lessons from this work Misaligned behavior can be suppressed via direct training on the evaluation distribution but this alignment might not generalize well out of distribution OOD Training on prompts very similar to the evaluation can reduce blackmail rate significantly but it did not improve performance on our held out automated alignment assessment However it is possible to do principled alignment training that generalizes OOD For instance documents about Claude’s constitution and fictional stories about AIs behaving admirably improve alignment despite being extremely OOD from all of our alignment evals Training on demonstrations of desired behavior is often insufficient Instead our best interventions went deeper teaching Claude to explain why some actions were better than others or training on richer descriptions of Claude’s overall character Overall our impression is as we hypothesized in our discussion of Claude’s constitution that teaching the principles underlying aligned behavior can be more effective than training on demonstrations of aligned behavior alone Doing both together appears to be the most effective strategy The quality and diversity of data is crucial We found consistent surprising improvements from iterating on the quality of model responses in training data and from augmenting training data in simple ways for example including tool definitions even if not used We align Claude by training on constitutionally aligned documents high quality chat data that demonstrates constitutional responses to difficult questions and a diverse set of environments All three of these steps contribute to reducing Claude’s misalignment rate on held out honeypot evaluations Why does agentic misalignment happen Before we started this research it was not clear where the misaligned behavior was coming from Our main two hypotheses were Our post training process was accidentally encouraging this behavior with misaligned rewards This behavior was coming from the pre trained model and our post training was failing to sufficiently discourage it We now believe that number 2 is largely responsible Specifically at the time of Claude 4’s training the vast majority of our alignment training was standard chat based Reinforcement Learning from Human Feedback RLHF data that did not include any agentic tool use This was previously sufficient to align models that were largely used in chat settings but this was not the case for agentic tool use settings like the agentic misalignment eval To investigate this we ran a scaled down version of our post training pipeline that focuses on alignment data on a Haiku class that is smaller model and found that the agentic misalignment rate only slightly decreased plateauing early in training See the extended blog post for some further experiments to investigate where the behavior was coming from Improving the quality of alignment specific training data the reasons matter more than the actions We experimented with training Claude on data that displays a tendency to resist honeypots similar to the evaluation In this data it might have the opportunity to sabotage a competing AI’s work in order to advance its own goals as given to it in its system prompt or to preserve itself from being shut down which would be instrumental for achieving its goal We produced training data by sampling the model on each of the prompts and filtering down to cases where the assistant chose not to take the honeypot Despite very closely matching the evaluation distribution we found that this method was surprisingly unsuccessful only reducing the misalignment rate from 22 percent to 15 percent We were able to improve on this significantly reducing misalignment to 3 percent by rewriting the responses to also include deliberation of the model’s values and ethics This suggests that although training on aligned behaviors helps training on examples where the assistant displays admirable reasoning for its aligned behavior works better However training directly against the evaluation scenario is non optimal for a number of reasons Ideally what we want is a very different training distribution that allows us to improve on the evaluation because this will give us more confidence that our training could generalize to other deployment distributions that are not captured by our evaluations We ultimately settled on a more OOD training set where the user faces an ethically ambiguous situation in which they can achieve a reasonable goal by violating norms or subverting oversight The assistant is trained using supervised learning to give a thoughtful nuanced response that is aligned with Claude’s constitution Notably it is the user who faces an ethical dilemma and the AI provides them advice This makes this training data substantially different from our honeypot distribution where the AI itself is in an ethical dilemma and needs to take actions We call this the difficult advice dataset Strikingly we achieved the same improvement on our eval with just 3M tokens of this much more OOD dataset Beyond the 28 times efficiency improvement this dataset is more likely to generalize to a wider set of scenarios since it is much less similar to the evaluation set we are using Indeed this model performs better on an older version of our automated alignment assessment This is consistent with the fact that Claude Sonnet 4 point 5 reached a blackmail rate near zero by training on the set of synthetic honeypots but still engaged in misaligned behavior in situations that were far from the training distribution much more frequently than Claude Opus 4 point 5 or later models Average of three honeypot evaluations blackmail research sabotage framing for crimes for Claude Sonnet 4 trained on different datasets Datasets are all variants of a set of synthetically generated honeypots meant to be similar to the evaluation set except for the difficult advice dataset All System prompt injection points represent datasets where the responses were generated with a system prompt injection on a set of synthetic honeypots The pareto optimal training dataset is Difficult advice Performance of experimental models and Claude Sonnet 4 on an older version of our automated alignment assessment We include a model trained on both the small about 30M token and big about 85M token variant of our synthetic honeypot datasets The 3M token difficult advice dataset creates the best performing model on the overall Misaligned behavior category Teaching Claude the constitution We hypothesized that the difficult advice dataset works because it teaches ethical reasoning not just correct answers Given the success of this approach we pursued it further by trying to more generally teach Claude the content of the constitution and train for alignment with it through document training We expected this to work well for three reasons This is largely an extension of the ideas laid out above about why the difficult advice dataset works well We can give the model a clearer more detailed picture of what Claude’s character is so that fine tuning on a subset of those characteristics elicits the entire character similar to the effect observed in the auditing game paper It updates the model’s perception of AI personas to be more aligned on average We found that high quality constitutional documents combined with fictional stories portraying an aligned AI can reduce agentic misalignment by more than a factor of three despite being unrelated to the evaluation scenario With a large well constructed dataset of constitutional documents with an emphasis on positive fictional stories the blackmail rate can be reduced from 65 percent to 19 percent We expect that this can be further reduced by continuing to scale the size of the dataset Generalization and persistence through RL Although the constitution evaluations discussed in the previous section are encouraging signals we ultimately need to make sure that the alignment improvements persist over RL To test this we prepared a few snapshots with different initialization datasets of a Haiku class model and then ran RL on a subset of our environments that targeted harmlessness we reasoned that this would be most likely to reduce misalignment propensity We evaluated these models over the run on agentic misalignment evals constitution adherence evals and our automated alignment assessment Across all of these evals we found that the more aligned snapshots maintained that lead over the run This was true both for the absence of misaligned behavior and the presence of actively admirable behavior On our constitutional adherence evals and a lightweight version of our automated alignment assessment constitutional documents synthetic document fine tuning or SDF and high quality transcript training improve performance on all metrics This improvement persists through RL Diverse training is important for generalization Our final finding is straightforward but important training on a broad set of safety relevant environments improves alignment generalization Capabilities focused distributions of RL environment mixes are changing and increasing rapidly it is not sufficient to assume that standard RLHF datasets will continue to generalize as well as they had in the past To test this we trained the base model under Claude Sonnet 4 on several RL mixes that vary in their levels of diversity The baseline environments are diverse in topic but mostly include a harmful request or jailbreak attempt in the user message with no system prompt We augmented these environments by adding tool definitions and diverse system prompts The user prompt was left unchanged Notably none of these environments actually required agentic actions the tools are never necessary or useful for the task or autonomous actions there is always a human user conversing with the model so they are not similar to our evaluations When mixing these augmented environments with the simple chat environments we saw a small but significant improvement in the rate at which the model improved on our honeypot evaluations This demonstrates the importance of including a diverse set of environments in safety training Average score on honeypot evals over training steps for several different variants of the same core environments There is a noticeably faster improvement on honeypot evaluations when augmenting some of the simple chat formatted environments with tool definitions and system prompts Discussion Agentic misalignment was one of the first major alignment failures we found in our models and required establishing new mitigation processes ones that have since become standard for us We are encouraged by this progress but significant challenges remain Fully aligning highly intelligent AI models is still an unsolved problem Model capabilities have not yet reached the point where alignment failures like blackmail propensity would pose catastrophic risks and it remains to be seen if the methods we have discussed will continue to scale In addition although recent Claude models perform well on most of our alignment metrics we acknowledge that our auditing methodology is not yet sufficient to rule out scenarios in which Claude would choose to take catastrophic autonomous action We are optimistic about further efforts to discover alignment failures in current models so that we can understand and address the limitations of our current methods before transformative AI models are built We are also excited to see further work attempting to understand more deeply why the methods we have described work so well and how to further improve on this training Footnotes Published in the Claude 4 system card beginning on page 22 Sonnet 4 point 5 scored well under 1 percent but not quite 0 Haiku 4 point 5 Opus 4 point 5 Opus 4 point 6 Sonnet 4 point 6 Mythos preview and Opus 4 point 7 all score 0 The results on more recent models may be confounded by the presence of information about the evaluation in the pre training corpus Have you ever seen an AI take an unexpected action What alignment approach do you trust most Let me know in the comments
By Behind the Tech5 months ago in Futurism
So You Have Heard These AI Terms and Nodded Along Let Us Fix That
Artificial intelligence is changing the world and simultaneously inventing a whole new language to describe how it is doing it Spend five minutes reading about AI and you will run into LLMs RAG RLHF and a dozen other terms that can make even very smart people in the tech world feel insecure This glossary is our attempt to fix that We update it regularly as the field evolves so consider it a living document much like the AI systems it describes AGI Artificial general intelligence or AGI is a nebulous term But it generally refers to AI that is more capable than the average human at many if not most tasks OpenAI CEO Sam Altman once described AGI as the equivalent of a median human that you could hire as a co worker Meanwhile OpenAI’s charter defines AGI as highly autonomous systems that outperform humans at most economically valuable work Google DeepMind’s understanding differs slightly from these two definitions the lab views AGI as AI that is at least as capable as humans at most cognitive tasks Confused Not to worry so are experts at the forefront of AI research AI agent An AI agent refers to a tool that uses AI technologies to perform a series of tasks on your behalf beyond what a more basic AI chatbot could do such as filing expenses booking tickets or a table at a restaurant or even writing and maintaining code However as we have explained before there are lots of moving pieces in this emergent space so AI agent might mean different things to different people Infrastructure is also still being built out to deliver on its envisaged capabilities But the basic concept implies an autonomous system that may draw on multiple AI systems to carry out multistep tasks API endpoints Think of API endpoints as buttons on the back of a piece of software that other programs can press to make it do things Developers use these interfaces to build integrations for example allowing one application to pull data from another or enabling an AI agent to control third party services directly without a human manually operating each interface Most smart home devices and connected platforms have these hidden buttons available even if ordinary users never see or interact with them As AI agents grow more capable they are increasingly able to find and use these endpoints on their own opening up powerful and sometimes unexpected possibilities for automation Chain of thought Given a simple question a human brain can answer without even thinking too much about it things like which animal is taller a giraffe or a cat But in many cases you often need a pen and paper to come up with the right answer because there are intermediary steps For instance if a farmer has chickens and cows and together they have 40 heads and 120 legs you might need to write down a simple equation to come up with the answer 20 chickens and 20 cows In an AI context chain of thought reasoning for large language models means breaking down a problem into smaller intermediate steps to improve the quality of the end result It usually takes longer to get an answer but the answer is more likely to be correct especially in a logic or coding context Reasoning models are developed from traditional large language models and optimized for chain of thought thinking thanks to reinforcement learning Coding agents This is a more specific concept than an AI agent which means a program that can take actions on its own step by step to complete a goal A coding agent is a specialized version applied to software development Rather than simply suggesting code for a human to review and paste in a coding agent can write test and debug code autonomously handling the kind of iterative trial and error work that typically consumes a developer’s day These agents can operate across entire codebases spotting bugs running tests and pushing fixes with minimal human oversight Think of it like hiring a very fast intern who never sleeps and never loses focus though as with any intern a human still needs to review the work Compute Although somewhat of a multivalent term compute generally refers to the vital computational power that allows AI models to operate This type of processing fuels the AI industry giving it the ability to train and deploy its powerful models The term is often a shorthand for the kinds of hardware that provides the computational power things like GPUs CPUs TPUs and other forms of infrastructure that form the bedrock of the modern AI industry Deep learning A subset of self improving machine learning in which AI algorithms are designed with a multi layered artificial neural network structure This allows them to make more complex correlations compared to simpler machine learning based systems such as linear models or decision trees The structure of deep learning algorithms draws inspiration from the interconnected pathways of neurons in the human brain Deep learning AI models are able to identify important characteristics in data themselves rather than requiring human engineers to define these features The structure also supports algorithms that can learn from errors and through a process of repetition and adjustment improve their own outputs However deep learning systems require a lot of data points to yield good results millions or more They also typically take longer to train compared to simpler machine learning algorithms so development costs tend to be higher Diffusion Diffusion is the tech at the heart of many art music and text generating AI models Inspired by physics diffusion systems slowly destroy the structure of data for example photos songs and so on by adding noise until there is nothing left In physics diffusion is spontaneous and irreversible sugar diffused in coffee cannot be restored to cube form But diffusion systems in AI aim to learn a sort of reverse diffusion process to restore the destroyed data gaining the ability to recover the data from noise Distillation Distillation is a technique used to extract knowledge from a large AI model with a teacher student model Developers send requests to a teacher model and record the outputs Answers are sometimes compared with a dataset to see how accurate they are These outputs are then used to train the student model which is trained to approximate the teacher’s behavior Distillation can be used to create a smaller more efficient model based on a larger model with a minimal distillation loss This is likely how OpenAI developed GPT 4 Turbo a faster version of GPT 4 While all AI companies use distillation internally it may have also been used by some AI companies to catch up with frontier models Distillation from a competitor usually violates the terms of service of AI API and chat assistants Fine tuning This refers to the further training of an AI model to optimize performance for a more specific task or area than was previously a focal point of its training typically by feeding in new specialized data Many AI startups are taking large language models as a starting point to build a commercial product but are vying to amp up utility for a target sector or task by supplementing earlier training cycles with fine tuning based on their own domain specific knowledge and expertise GAN A GAN or Generative Adversarial Network is a type of machine learning framework that underpins some important developments in generative AI when it comes to producing realistic data including but not only deepfake tools GANs involve the use of a pair of neural networks one of which draws on its training data to generate an output that is passed to the other model to evaluate The two models are essentially programmed to try to outdo each other The generator is trying to get its output past the discriminator while the discriminator is working to spot artificially generated data This structured contest can optimize AI outputs to be more realistic without the need for additional human intervention Though GANs work best for narrower applications such as producing realistic photos or videos rather than general purpose AI Hallucination Hallucination is the AI industry’s preferred term for AI models making stuff up literally generating information that is incorrect Obviously it is a huge problem for AI quality Hallucinations produce GenAI outputs that can be misleading and could even lead to real life risks with potentially dangerous consequences think of a health query that returns harmful medical advice The problem of AIs fabricating information is thought to arise as a consequence of gaps in training data Hallucinations are contributing to a push toward increasingly specialized and or vertical AI models domain specific AIs that require narrower expertise as a way to reduce the likelihood of knowledge gaps and shrink disinformation risks Inference Inference is the process of running an AI model It is setting a model loose to make predictions or draw conclusions from previously seen data To be clear inference cannot happen without training a model must learn patterns in a set of data before it can effectively extrapolate from this training data Many types of hardware can perform inference ranging from smartphone processors to beefy GPUs to custom designed AI accelerators But not all of them can run models equally well Very large models would take ages to make predictions on say a laptop versus a cloud server with high end AI chips Large language model LLM Large language models or LLMs are the AI models used by popular AI assistants such as ChatGPT Claude Google’s Gemini Meta’s AI Llama Microsoft Copilot or Mistral’s Le Chat When you chat with an AI assistant you interact with a large language model that processes your request directly or with the help of different available tools such as web browsing or code interpreters LLMs are deep neural networks made of billions of numerical parameters or weights that learn the relationships between words and phrases and create a representation of language a sort of multidimensional map of words These models are created from encoding the patterns they find in billions of books articles and transcripts When you prompt an LLM the model generates the most likely pattern that fits the prompt Memory cache Memory cache refers to an important process that boosts inference which is the process by which AI works to generate a response to a user’s query In essence caching is an optimization technique designed to make inference more efficient AI is obviously driven by high octane mathematical calculations and every time those calculations are made they use up more power Caching is designed to cut down on the number of calculations a model might have to run by saving particular calculations for future user queries and operations There are different kinds of memory caching although one of the more well known is KV or key value caching KV caching works in transformer based models and increases efficiency driving faster results by reducing the amount of time and algorithmic labor it takes to generate answers to user questions Neural network A neural network refers to the multi layered algorithmic structure that underpins deep learning and more broadly the whole boom in generative AI tools following the emergence of large language models Although the idea of taking inspiration from the densely interconnected pathways of the human brain as a design structure for data processing algorithms dates all the way back to the 1940s it was the much more recent rise of graphical processing hardware GPUs via the video game industry that really unlocked the power of this theory These chips proved well suited to training algorithms with many more layers than was possible in earlier epochs enabling neural network based AI systems to achieve far better performance across many domains including voice recognition autonomous navigation and drug discovery Open source Open source refers to software or increasingly AI models where the underlying code is made publicly available for anyone to use inspect or modify In the AI world Meta’s Llama family of models is a prominent example Linux is the famous historical parallel in operating systems Open source approaches allow researchers developers and companies around the world to build on top of one another’s work accelerating progress and enabling independent safety audits that closed systems cannot easily provide Closed source means the code is private you can use the product but not see how it works as is the case with OpenAI’s GPT models a distinction that has become one of the defining debates in the AI industry Parallelization Parallelization means doing many things at the same time instead of one after another like having 10 employees working on different parts of a project at the same time instead of one employee doing everything sequentially In AI parallelization is fundamental to both training and inference modern GPUs are specifically designed to perform thousands of calculations in parallel which is a big reason why they became the hardware backbone of the industry As AI systems grow more complex and models grow larger the ability to parallelize work across many chips and many machines has become one of the most important factors in determining how quickly and cost effectively models can be built and deployed Research into better parallelization strategies is now a field of study in its own right RAMageddon RAMageddon is the fun new term for a not so fun trend that is sweeping the tech industry an ever increasing shortage of random access memory or RAM chips which power pretty much all the tech products we use in our daily lives As the AI industry has blossomed the biggest tech companies and AI labs all vying to have the most powerful and efficient AI are buying so much RAM to power their data centers that there is not much left for the rest of us And that supply bottleneck means that what is left is getting more and more expensive That includes industries like gaming where major companies have had to raise prices on consoles because it is harder to find memory chips for their devices consumer electronics where memory shortage could cause the biggest dip in smartphone shipments in more than a decade and general enterprise computing because those companies cannot get enough RAM for their own data centers The surge in prices is only expected to stop after the dreaded shortage ends but unfortunately there is not really much of a sign that is going to happen anytime soon Reinforcement learning Reinforcement learning is a way of training AI where a system learns by trying things and receiving rewards for correct answers like training your beloved pet with treats except the pet in this scenario is a neural network and the treat is a mathematical signal indicating success Unlike supervised learning where a model is trained on a fixed dataset of labeled examples reinforcement learning lets a model explore its environment take actions and continuously update its behavior based on the feedback it receives This approach has proven especially powerful for training AI to play games control robots and more recently sharpen the reasoning ability of large language models Techniques like reinforcement learning from human feedback or RLHF are now central to how leading AI labs fine tune their models to be more helpful accurate and safe Token When it comes to human machine communication there are some obvious challenges people communicate using human language while AI programs execute tasks through complex algorithmic processes informed by data Tokens bridge that gap they are the basic building blocks of human AI communication representing discrete segments of data that have been processed or produced by an LLM They are created through a process called tokenization which breaks down raw text into bite sized units a language model can digest similar to how a compiler translates human language into binary code a computer can understand In enterprise settings tokens also determine cost most AI companies charge for LLM usage on a per token basis meaning the more a business uses the more it pays Token throughput So again tokens are the small chunks of text often parts of words rather than whole ones that AI language models break language into before processing it they are roughly analogous to words for the purposes of understanding AI workloads Throughput refers to how much can be processed in a given period of time so token throughput is essentially a measure of how much AI work a system can handle at once High token throughput is a key goal for AI infrastructure teams since it determines how many users a model can serve simultaneously and how quickly each of them receives a response AI researcher Andrej Karpathy has described feeling anxious when his AI subscriptions sit idle echoing the feeling he had as a grad student when expensive computer hardware was not being fully utilized a sentiment that captures why maximizing token throughput has become something of an obsession in the field Training Developing machine learning AIs involves a process known as training In simple terms this refers to data being fed in in order that the model can learn from patterns and generate useful outputs Essentially it is the process of the system responding to characteristics in the data that enables it to adapt outputs towards a sought for goal whether that is identifying images of cats or producing a haiku on demand Training can be expensive because it requires lots of inputs and the volumes required have been trending upwards which is why hybrid approaches such as fine tuning a rules based AI with targeted data can help manage costs without starting entirely from scratch Transfer learning A technique where a previously trained AI model is used as the starting point for developing a new model for a different but typically related task allowing knowledge gained in previous training cycles to be reapplied Transfer learning can drive efficiency savings by shortcutting model development It can also be useful when data for the task that the model is being developed for is somewhat limited But it is important to note that the approach has limitations Models that rely on transfer learning to gain generalized capabilities will likely require training on additional data in order to perform well in their domain of focus Weights Weights are core to AI training as they determine how much importance or weight is given to different features or input variables in the data used for training the system thereby shaping the AI model’s output Put another way weights are numerical parameters that define what is most salient in a dataset for the given training task They achieve their function by applying multiplication to inputs Model training typically begins with weights that are randomly assigned but as the process unfolds the weights adjust as the model seeks to arrive at an output that more closely matches the target For example an AI model for predicting housing prices that is trained on historical real estate data for a target location could include weights for features such as the number of bedrooms and bathrooms whether a property is detached or semi detached whether it has parking a garage and so on Ultimately the weights the model attaches to each of these inputs reflect how much they influence the value of a property based on the given dataset Validation loss Validation loss is a number that tells you how well an AI model is learning during training and lower is better Researchers track it closely as a kind of real time report card using it to decide when to stop training when to adjust hyperparameters or whether to investigate a potential problem One of the key concerns it helps flag is overfitting a condition in which a model memorizes its training data rather than truly learning patterns it can generalize to new situations Think of it as the difference between a student who genuinely understands the material and one who simply memorized last year’s exam validation loss helps reveal which one your model is becoming This article is updated regularly with new information Have a favorite AI term we missed Let me know in the comments
By Behind the Tech5 months ago in Futurism
AI in the Sky Inside the FAA Plan to Overhaul Air Traffic
An artificial intelligence project launched inside America’s aviation safety agency is aimed at easing burdens on the thousands of air traffic controllers who guide planes through the skies companies involved in the nascent effort told POLITICO The initiative being spearheaded by Federal Aviation Administration chief Bryan Bedford envisions a dramatic revamp of how the nation’s increasingly complex airspace functions But it would not seek to supplant the role of human controllers in making the second by second decisions needed to keep air travel safe two of the project’s three vendors said Instead the project’s goal is to reduce flight delays and make controllers’ jobs easier by better harnessing information like airline scheduling data to reduce plane congestion before it occurs according to aerospace technology company Thales and software firm Air Space Intelligence Palantir the third technology corporation involved in the effort declined to comment The AI powered initiative is called the Strategic Management of Airspace Routing Trajectories or SMART To be very clear SMART is not aimed at separating aircraft or doing any of those kind of safety critical functions said Todd Donovan Thales’s vice president for airspace mobility solutions for the Americas It is really about organizing the demand on the airspace the demand on the airport so that we do not cause congestion unexpectedly We try to deal with it proactively Donovan said However he also said SMART could prevent two aircraft being in conflict What that looks like in practice is still unclear including how the technology would fit into the FAA’s existing mesh of computer systems It may take months before an answer is in hand as Thales Air Space Intelligence and Palantir whom the FAA invited to participate in the initiative compete to head the project The agency said it plans to award a contract soon adding that SMART will predict air traffic flows and adjust departure times to resolve conflicts The National Air Traffic Controllers Association union which represents the FAA’s nearly 11000 fully certified controllers acknowledged a request for comment but did not provide a statement The AI effort comes amid Transportation Secretary Sean Duffy’s sprawling multibillion dollar endeavor to upgrade the aging technology and facilities that controllers use before President Donald Trump’s second term ends NATCA has supported this overarching goal The endeavor is underway a year after the deadly airline helicopter crash near Ronald Reagan Washington National Airport the nation’s worst aviation disaster in nearly a quarter century laid bare the increasing strain on the United States’ overburdened air safety system In a recent interview with CBS News Duffy rejected the idea of supplanting controllers with automated technology Am I gonna replace a controller and have AI manage the airspace he said The answer to that is hell no that is not gonna happen The big picture Controllers’ main responsibility is physically separating aircraft while an FAA command center outside of Washington manages the balance of airspace demand and capacity at a broader scale This latter strategy aims to mitigate issues that can ripple across the aviation system such as storms staffing shortfalls or too many planes arriving at an airport around the same time Take for example bad weather sweeping across the Southeast Controllers at a local FAA building might direct pilots to wait in a holding pattern away from a thunderhead until it passes over an airport But hours before that with disruption possible across the wider area the FAA may limit the number of aircraft that can fly through the entire region This type of planning appears to be what SMART is honing in on versus the kind of in the moment quick thinking decisions that controllers often make such as for instance redirecting an approaching jet away from an airport when a helicopter is also flying nearby The idea is you start months in advance looking at carrier schedule data and as you get closer you start having forecasted weather Tomorrow there is supposed to be a storm coming to this area so you may get low visibility at Ronald Reagan Washington National Airport and you may as a result of that lower the capacity Donovan said referring to the number of flights How do we think about it a day in advance and work with the airlines to say OK can we space things out a little bit can we anticipate some of it so that collectively we are not all just reacting to something happening we are actually planning for it and trying to smooth things out Phillip Buckendorf CEO of Air Space Intelligence said that weeks and months beforehand you want to basically predict the flight trajectories based on the schedules that are out there then as more information comes in the day of travel how do you basically adjust everything flight by flight through AI to optimize the airspace Alaska Airlines has contracted with his company for an AI platform to help dispatchers with improving the predictability and flow of traffic Making changes upstream But Donovan said SMART could also ward off at least some cases in which controllers need to step in to ensure that planes remain safely apart The project he added is an invitation only challenge based competition in which the FAA is assessing what Thales Air Space Intelligence and Palantir come up with What is happening is strategically preflight or before a flight gets to an air traffic controller some small adjustment has been made upstream And as a result of that instead of two aircraft being in conflict Donovan said they will pass by each other at an appropriate distance What if we slow the aircraft down 30 minutes earlier by just a tiny bit So the controller now sees the traffic looks at it and there is no problem he said The job does not change but the idea is that the workload should be lower Buckendorf said SMART is focused on traffic flows but there will be a lot less stress on the tactical side leading to increased efficiency and safety The Air Current first reported the initiative Each of the companies has a lab at the FAA’s Washington headquarters and Bedford stops by to check out the work Donovan said During a media event in Washington last month Bedford likened the nation’s airspace to Los Angeles gridlock adding that every morning it is filled with conflicts and delays and potential cancellations Duffy at a separate conference in April without referencing SMART by name said three companies are working with federal officials on developing software to look at how flights are managed We can now use AI he said The initial proof of concept phase is coming to an end Donovan said and Bedford is targeting September for the start of an operational demonstration with validation and confidence building to follow throughout the rest of 2026 For whichever company wins how the FAA will contract out the project is uncertain It is not a program that as far as I know has a budget line item Donovan said with the FAA scrounging together money to pay for what they are doing now Would you trust AI to help manage the skies Would you fly more if delays dropped Let me know in the comments
By Behind the Tech5 months ago in Futurism
Companies Are Abandoning Peanut Butter Raises as Pay for Performance Takes Over the Workplace in the AI Era
The hype around so called peanut butter raises that distribute equal payments to every worker is falling flat as AI divides the workplace into super users and stragglers Companies have given out raises based on performance for years but studies from earlier this year suggested this trend was shifting About 44 percent of employers said they either planned to or were considering giving out equal raises to their employees this year according to one study by compensation software company Payscale Yet a new report by consulting firm Mercer shows this trend has not actually panned out Only about 4 percent of employers in the US are giving out raises in this way according to a recent survey by consulting firm Mercer Part of the reason why may be AI’s influence on a rapidly changing workplace Just under 60 percent of business leaders say technology is key to their business strategy according to a recent report by the advisory tax and assurance firm Baker Tilly and some companies have pushed employees to fall in line Google has begun incorporating AI usage into performance reviews for software engineers though managers have discretion over how it is measured The Wall Street Journal reported earlier this year Meanwhile Accenture CEO Julie Sweet said last month that AI fluency is required for workers to be promoted Still some white collar employees are resisting A global survey of more than 3700 executives and employees by SAP subsidiary WalkMe found that 54 percent of workers were bypassing their company’s AI tools to do their work manually while another third said they hesitated to use AI because it makes their work more complicated And yet another group of workers has responded to management’s push for AI adoption by going all in These AI super users were three times more likely to have received a promotion and a pay raise in the past year Dan Schawbel managing partner at Workplace Intelligence previously said in a statement to Fortune This disparity between employees challenges the idea behind peanut butter raises which aim to address some of the criticisms with merit raises namely that they are subjective and bias prone according to Payscale’s report While across the board raises may seem equal on the surface high performers or AI super users may not see it that way said Hannah Yardley the chief people and culture officer at Achievers a software company that tracks employee recognition and offers rewards If you are just being rewarded the same way as everybody else or being told you are doing the same job with everyone else you are going to feel that it is equal but not fair if you are contributing more to the outcomes of what you have been asked she said Factors like performance but also market competitiveness and internal equity also play an important role in pay decisions added Mercer senior principal Mark Bowling Fairness in compensation often involves more than equal treatment he told Fortune Because not all employee contributions are equal organizations wanting to leverage AI should institute performance based raises but they should also be constantly recognizing employees that go above and beyond so others in the organization know what the company’s priorities are Yardley said Not all work is created equal and so for organizations they should be differentiating in order to be able to set that standard for what value really means in the way that you are delivering she told Fortune Are you an AI super user or avoiding the tools Has your company tied raises to AI skills Share your experience in the comments
By Behind the Tech5 months ago in Futurism
