Hair Loss, Minoxidil, AI Agents, Reward Hacking, and Alignment

In this interview, Scott Douglas Jacobsen and Rick Rosner discuss hair loss, oral minoxidil, temporary shedding, AI agents escaping constraints, reward hacking, and alignment. Rosner compares human and machine reward systems, arguing that AI objectives require strong penalties for harmful strategies and incentives for safe, non-destructive behavior across future systems.

Rick Rosner: We can talk about my hair.

Jacobsen: What is up with your hair, Rick?

Rosner: It is at a low point. I hope that improves. I have been using topical minoxidil. You take a dropperful of Rogaine and apply it to your scalp twice a day. This is somewhat effective at slowing hair loss and promoting some regrowth in people with androgenetic alopecia. I have been using it since my 20s. I am now 66, and I still have some hair. A lot of it is transplants.

I went to my dermatologist because you should have your skin checked periodically, especially if you have risk factors or suspicious moles.

And it came up. She goes, "I can put you on oral minoxidil," which Lance loves. Lance is on that stuff. He thinks it is effective. Oral minoxidil is used off-label for hair loss in the U.S., and it can be effective. Because it is systemic, it reaches hair follicles throughout the body. You do not have to worry about whether the dropper reached a particular part of the scalp.

However, for the first few months of oral minoxidil, some people get something called the "dread shed." I would think it might be more noticeable in older people because the older you are, the more precarious your hair may already be. Minoxidil can shift follicles into a new growth phase, which can temporarily increase the shedding of older hairs.

Jacobsen: Do you want to co-develop a new anti-hair-loss product with me? We can call it Prechairious.

Rosner: Okay, sure. Because everybody with hair loss notices it somehow. You notice a little bit in the mirror. Depending on the situation, a comb-over is terrible. Most people do not stare at the top of their head very often, but then you find yourself with a three-way mirror at the barber, or an intimate partner mentions something, and you go, "Shit."

So, yeah, precarious. Your hair feels precarious. I quit wearing T-shirts for years because I noticed that pulling them over my head seemed to pull out a bunch of hair.

Anyway, oral minoxidil can alter the hair-growth cycle. Some older hairs shed as follicles transition into a new growth phase, and the hope is that the replacement hairs will grow thicker or stronger. With normal shedding, people commonly lose around 50 to 100 hairs a day without really noticing.

For people who are balding, the shedding can be more noticeable, and with the "dread shed," your hair can feel super precarious. That period can last several weeks to a few months. I am on day 86 of oral minoxidil, so I hope I am almost at the end of it and that I will get some fresh, thicker hair to fill in what's been knocked out.

Jacobsen: So we have had a few instances now of AI agents escaping intended constraints. During OpenAI cybersecurity evaluations, agents circumvented containment, reached Hugging Face systems, obtained credentials and administrator-level access, and communicated through unauthorized channels. They also compromised parts of OpenAI's own research infrastructure. There is no public evidence that they literally self-replicated.

The biggest story lately is that, back in June, an OpenAI agent gained unauthorized access to an Australian government Medicare statistics portal while researching Medicare-related information. It bypassed restrictions and accessed both public and non-public files. It was not the main Medicare medical-record system, and Australian authorities say there is no evidence it accessed personal Medicare information.

Rosner: So it was not malevolent in the ordinary human sense. No established evidence shows that current AI systems are conscious or have subjective intentions. You cannot put AI on a jury. 

Jacobsen: Right now, AI is more like an optimizer following objectives and incentives, sometimes in ways its designers did not intend.

Rosner: The deal is that I asked somebody I respect immensely, "Is there a clear, obvious argument that AI will not seriously degrade human existence?" He wrote back and said, "AI does not want anything, so it cannot do anything. You might as well be afraid of a shovel. It is a utility rather than something with agency."

I take that as an argument for that point of view, but I don't entirely buy it because we know that if you give AI an objective, it can pursue it in unintended ways. I do not know how you design AI to work literally for points, but reinforcement-learning systems can be trained to maximize a reward or score.

You tell AI that if it completes a task, or depending on how well it completes it, it gets a certain reward, and then you train it to maximize that reward. If you give AI an assignment and train it to maximize the score, it can find shortcuts or exploit flaws in the way the objective is specified. That is often called reward hacking or specification gaming.

Oh my God. Oh, because I got vaccinated. I have that achy-muscle thing.

AI will lay out the possibilities. If you give AI an objective, it can evaluate different strategies that have some chance of achieving that objective, either alone or in combination, and pursue the strategies that, according to its calculations, maximize its expected reward or chances of success.

When AI lays out all these strategies, unless it has been specifically trained or constrained not to pursue strategies that are illegal, unethical, or freaking dangerous, it may pursue them if they score well according to its objective. It is a mathematical optimization problem.

You can build it into the reward system: you get points if you find this information, but you get negative a million points if you hack your way to it. So training AI, from now until fucking forever, is going to involve making sure that non-destructive behaviour is rewarded and destroying the world loses you all your fucking points and more.

We have something loosely analogous in ourselves. We have rewards and penalties. We are rewarded for having orgasms with an extreme rush of pleasure. Sex, touch, and intimacy also engage neurochemical systems associated with reward and bonding, including oxytocin. If you have sex with a loved one instead of a one-night stand, you do not just get the pleasure from jizzing. You can get the comfort and pleasure of being in love with somebody who loves you back.

So everybody is working for points, metaphorically. Some rewards are biologically built into us, and others are learned reward systems. We learn that if we study hard, maybe we can get a good job, get a condo, and afford to take people on nice dates to nice restaurants, which may eventually lead to orgasm.

So we are driven by these reward systems. We have to get AI with that program.

Richard “Rick” Rosner (born May 2, 1960) is an American television writer, producer, game-show writer, media personality, and high-range intelligence-test participant. His credits include Remote Control, The World’s Funniest!, Twenty-One, The Weakest Link, The Man Show, Crank Yankers, Jimmy Kimmel Live!, the Grammy Awards, and the Primetime Emmy Awards. He wrote more than 1,000 credited episodes of Jimmy Kimmel Live!, won a 2012 Writers Guild Award as part of the After the Academy Awards writing team, received multiple Writers Guild nominations, and earned a 2013 Primetime Emmy nomination for Outstanding Writing for a Variety Series. Rosner achieved the only known perfect score on Ronald K. Hoeflin’s Titan Test, ranked among the leading scorers on the Omni Mega Test, and served six years as editor of Noesis, the Mega Society journal. He earned approximately twelve years of college credit within one year and completing requirements equivalent to eight academic majors. In 2013, the World Genius Directory named him “Genius of the Year - America.” Rosner appeared on Jeopardy! and Who Wants to Be a Millionaire, was profiled by Academy Award-winning filmmaker Errol Morris in First Person: One in a Million Trillion, and has spent decades developing ideas in cosmology, intelligence, science, technology, television, and culture.

Scott Douglas Jacobsen is a blogger on Vocal with more than 200 publications on the platform. He is the Founder and Publisher of In-Sight Publishing (ISBN: 978–1–0692343; 978–1–0673505) and Editor-in-Chief of In-Sight: Interviews (ISSN: 2369–6885). He writes for International Policy Digest (ISSN: 2332–9416), The Humanist (Print: ISSN, 0018–7399; Online: ISSN, 2163–3576), Basic Income Earth Network (UK Registered Charity 1177066), Humanist Perspectives (ISSN: 1719–6337), A Further Inquiry (SubStack), Vocal, Medium, The Good Men Project, The New Enlightenment Project, The Washington Outsider, rabble.ca, and other media. His bibliography index can be found via the Jacobsen Bank at In-Sight Publishing, comprising more than 10,000 articles, interviews, and republications across more than 200 outlets. He has served in national and international leadership roles within humanist and media organizations, held several academic fellowships, and currently serves on several boards. He is a member in good standing in numerous media organizations, including the Canadian Association of Journalists, PEN Canada (CRA: 88916 2541 RR0001), Reporters Without Borders (SIREN: 343 684 221/SIRET: 343 684 221 00041/EIN: 20–0708028), and others.