Skip to content

kbps

LLMs make good analysts, bad oracles

I’m not an AI apologist by any means, but I’m frustrated by the muddled way LLMs have been marketed, portrayed, and used. I want to focus on the utility of them here, rather than the moral or legal implications of using copyrighted content to feed their corpora.

One of the first things we started doing when ChatGPT became public was, naturally, asking it questions. And for the most part, it gave us some pretty good answers.

But as has been widely demonstrated recently — as Google, Meta, and others have begun grafting “AI” onto their search results — is just how wrong it can get things, and it has had us asking: Is this really ready for widespread adoption? Asking for arbitrary bits of information from an LLM’s entire corpus of text — like Google’s and Bing’s smart summaries — is demonstrably, hilariously, and sometimes dangerously flawed.

Over the last couple years, I haven’t really heard much from OpenAI themselves about what we are supposed to be using ChatGPT for. They seem more interested in creating the technology — which no one could seriously doubt is impressive — than in finding real-world applications for it. The announcement for ChatGPT didn’t tell us what to do with it (though it did emphasize that the tool can be expected to product false information).

I think the misconception about what ChatGPT is purported to be good at can be attributed to the name and the UI. A chat-based interface to something called “ChatGPT” implies that the product is something it isn’t. It’s technically impressive, of course, and makes for a good demo. But chat doesn’t play to its strengths.

The reason any given LLM is even able to wager a guess at a general knowledge question is the corpus of text it’s been trained on. But producing answers to general knowledge questions is a side-effect of this training, not its purpose. It isn’t being fed an “encyclopedia module” that it classifies as facts about the world, followed by a “cookbook module” that it classifies as ways to prepare food. It was designed to produce believable language, not accurate language.

Where it does excel, however, is at coming to conclusions about narrow inputs. Things like Amazon’s review summaries; YouTube’s new grouping of comments by “topic”; or WordPress’s AI Feedback — these take specific streams of text and are tasked with returning feedback or summaries about them, and seem to work pretty well and have real utility.

These examples demonstrate two similar but distinct facets of LLMs: Their role as general knowledge engines, or “oracles,” and as input processing engines, or “analysts.” When we ask ChatGPT (or Google, or Meta) how many rocks we should eat per day, we are expecting it to behave as an oracle. When we ask it to summarize the plot of a short story or give us advice for improving our resume, we are expecting it behave as an analyst.

Signs point to Apple using LLMs primarily as analysts in the features to be announced at today’s WWDC, processing finite chunks of data into something else, rather than snatching arbitrary knowledge out of the LLM ether.

The allure of ChatGPT as an oracle is of course hard to resist. But I think if we recognize these two functions as separate, and focus on LLMs capabilities as analysts, we can wring some value out of them. (Their environmental impact notwithstanding.)

One Response

Apple TV+’s takeover of the Apple TV app Home Screen

Part of what used to be great about the Apple TV is that — unlike an Amazon Fire TV, for instance — Apple didn’t really have specific content it was trying to push on you. Sure, it might suggest that you rent or buy a movie from iTunes instead of from Amazon; and it would be safe to assume it might only recommend content that was available on iTunes, but practically that included everything.

Amazon, on the other hand, since they had gotten into the original content market, would be incentivized to give priority to their TV shows over others. Is my Amazon Fire TV recommending Sneaky Pete to me because it’s similar to stuff I watch, or because they stand to gain from it?

When Apple TV+ was announced, I worried that Apple’s having a stake in what specific content I watch would taint the Apple TV experience.

Boy, has it.

Below is a full view of the contents of the macOS TV app. The TV app on the Apple TV streaming box is roughly identical.

Continue reading

Leave a Comment

The war in Elon’s head

Everybody’s been talking about Elon’s “go fuck yourself” moment from DealBook, and the expletive is what is making most of the headlines. From a distance it could be easy to see this as just a petulant outburst from someone upset with the financial struggles of his company, but considering what he went on to say — and has said in the past — it becomes clear that this is about something else to him.

Elon: What this advertising boycott is gonna do, it’s gonna kill the company. And the whole world will know that those advertisers killed the company. And we will document it in great detail.

Andrew: But those advertisers, I imagine, are gonna say, “We didn’t kill the company.”

Elon: Oh yeah? Tell it to Earth.

Andrew: But they’re gonna say, Elon, that you killed the company because you said these things and that they were inappropriate things and that they didn’t feel comfortable on the platform.

Elon: And let’s see how Earth responds to that. We’ll both make our cases and we’ll see what the outcome is.

Andrew: What are the economics of that for you? You have enormous resources, so you can actually keep this company going for a very long time. Would you keep it going for a long time if there were no advertising?

Elon: If the company fails because of an advertiser boycott, it will fail because of an advertiser boycott. And that will be what bankrupted the company, and that’s what everybody on Earth will know. Then it’ll be gone. And it’ll be gone because of an advertiser boycott.

Andrew: But you recognize that some of those people are gonna say that they didn’t feel comfortable on the platform. And I just wonder, and ask you, and think about that for a second–

Elon: Tell it to the judge.

Andrew: But the judge is gonna be–

Elon: The judge is the public.

Something really struck me about the language he was using here. “Blackmail”? “Earth”? “We will document it in great detail”? Blackmail him to do what? What does he imagine this is about?

Of course, one way to read this is that he is shifting blame from himself to advertisers for the potential failure of X, and I’m sure that’s part of it. But he also seems to deeply believe that X is a heroic and final bulwark against creeping Orwellianism from politically correct elites.

To him, the disappearance of Twitter is a threat not just to himself or to the satisfied users of the social network, but an existential threat to “Earth.”

The way this plays out in his mind is that advertisers kill X, and because their abandonment of the platform will have been “documented in great detail,” Earth — Earth itself, not just the users of X — will retaliate. “Free speech” will die, there will be no medium left for expressing yourself without censorship, and the societal cost of this will be so great that these advertisers will themselves face extinction.

This hyperbolic view of things has been on display before, as when advertisers started expressing caution when he first took over in November 2022:

“They’re trying to destroy free speech in America.” Note that he doesn’t believe “free speech” will simply be an unfortunate casualty of the passive cowardice of advertisers fleeing X; he believes that advertisers’ fleeing X is a deliberate attempt to “destroy free speech.” They are invested in the death of X so that they can further their agenda. X is a threat to them, and they are trying to end it.

The “blackmail” he referred to at DealBook is, to his mind, blackmail to join their program of speech suppression. The elites want people to feel afraid to say certain things, and they want X and Elon on their team. By denying them this, he is martyring himself for the preservation of intellectual freedom for our entire species.

He has a messiah complex, and curiously it extends beyond his electric cars and plans to colonize Mars. Even a dumb social network is about his role in the fate of the planet.

I’m also amused by — and I haven’t seen any mention of this — the fact that Elon clearly seems to expect some kind of laughter/applause after his “go fuck yourself” line. He sees himself as a populist, and has gotten so used to being constantly fellated by his X subscribers that it shocks and confuses him when he isn’t showered with adulation for his irreverent “antics.”

One Response

The Monetized Web

Nobody loves a paywall, but everybody loves Substack

Increasingly, it feels like paid memberships for web content are not only a viable alternative to surveillance-driven ad revenue, but one that readers are eager to embrace. The success of Substack demonstrates this. This success is often framed as a preference for reading in the inbox rather than on the web, which is some feat considering how much people have come to loathe email in general over the last couple decades.

But I don’t think it’s the inbox per se that people like; it’s that the inbox gives people what the web used to — and no longer does, but could — give them: an ad-free and distraction-free reading experience. Medium was supposed to do this, but has, perhaps predictably, caved and started showing upsell popups and “related content” sidebars all over its article pages to get your money and keep you on the site.

Patreon, like Substack, has seen a lot of success, but for whatever reason isn’t really thought of as a place for longform writing. Its content tends to be audio, video, art, and access to Discord communities.

Continue reading

One Response

Apple’s incomplete pronoun fields

In iOS 17, you are now able to add pronouns to contacts, including your own contact card. This is a good thing, but at least one important pronoun case is missing from the “English” options.

The cases included are:

  1. Subjective (“Yesterday, she went outside.”)
  2. Objective (“I went with her.”)
  3. Possessive pronoun (“The idea was hers.”)

These three cases mirror the common “she/her/hers” structure used to communicate pronouns on social platforms like Twitter and Zoom; as such, they may seem complete, but they aren’t.

The missing case is the possessive adjective: “It was her idea.”

This may seem redundant, because in the declension of feminine pronouns, the possessive adjective is the same as the objective: “her.”

But in masculine pronouns, the possessive adjective is the same as the possessive pronoun: “his” (as in, “It was his idea”).

CaseFeminineMasculine
SubjectiveSheHe
ObjectiveHerHim
Possessive pronounHersHis
Possessive adjectiveHerHis

This isn’t a problem when interacting with people — we know how to decline the common feminine and masculine pronouns, so we know which form to use for the possessive adjective.

But because the iOS 17 UI doesn’t have a field for the possessive adjective, the OS and apps that have access to these fields — which have to behave programmatically — can’t know what to use for that case.

If I’m writing an app and I want the UI to say, “Would you like to call her?” in reference to some contact of yours, I can use the objective pronoun field. But if I want the UI to say “Her birthday is coming up” (the possessive adjective), I don’t have the necessary information.

I can try to guess, based on what I know about the declension of these common pronouns. But what about uncommon ones, like Ze or Xe? Anything I want to use will have to be hard-coded.

This is not even to mention the reflexive case — “She was proud of herself” — which is entirely absent.

Leave a Comment

MDN’s rogue definitions of <b> and <i>

From 1993 through roughly 2008, the <b> and <i> tags in HTML meant “bold” and “italic,” respectively. Using those tags will still, in 2023, cause most (all?) browsers to render text with either a bold font weight or an italic font style, but the tags no longer “mean” that. It’s now more correct to consider it a coincidence that browsers represent <b> as Bold and <i> as Italic; they may just as well be <y> and <r>.

The emphasis on both (a) semantic HTML and (b) backwards compatibility means that, as stated by the W3C themselves:

The b and i elements are widely used — it is better to give them good default rendering for various media including aural than to try to ban them.

So: What to do with those letters? <b> can’t mean “bold,” and <i> can’t mean “italic.” What do they mean?

Continue reading

Leave a Comment

How to get Jetpack’s “Writing Prompts” in WordPress

The WordPress Jetpack plugin recently (and experimentally) added “Writing Prompts” like those seen on WordPress.com-hosted sites since…I wanna say 2020?

I’ve been thinking about getting more into “Personal Blogging” elsewhere, and I figured these would be useful in getting words out.

In order to get these prompts on a self-hosted WordPress site, you need to do three things:

  1. Upgrade to JetPack 11.7 or higher
  2. Add the following line of code to your wp-config.php file:
    define( 'JETPACK_EXPERIMENTAL_BLOCKS', true );
  3. Go to “Settings > Writing” in WordPress admin, and check the box labeled “Show a writing prompt when starting a new post.”

I don’t know when this is supposed to become available without Step (2), but at that point it might be a good idea to turn off Experimental Blocks by removing that line.

2 Responses

“The Moon” Is Wrong

Three years ago, in December 2019, a tweet went viral posting a link to a YouTube video in which all the lyrics to Paul McCartney’s “Wonderful Christmastime” had been changed to “The moon is right.”

Two days later, another, far more viral tweet joked that the lyrics can be read as the interruption of an act of witchcraft. (This joke has been cribbed in countless subsequent social media posts.)

I was immediately skeptical at the hearing of the lyric as “moon” rather than “mood,” but it was helpfully pointed out to me that the official @PaulMcCartney Twitter account had indeed posted the lyrics as “moon.” (Original tweet link, Wayback archive)

Last year, in December 2021, @PaulMcCartney again tweeted the opening lyrics (backup link), but this time with “mood” rather than “moon.”

I began to hunt more seriously for answers.

That same winter I bought both physical and digital copies of the expanded edition of McCartney II. This album does include the song, but turns out not to have the song’s lyrics in its liner notes.

Unhelpfully, the song was originally released as a 45RPM single, with no printed lyrics.

I discovered that the official Paul McCartney YouTube channel had posted the music video in 2019, and its captions read “The mood is right.”

Liveright published “Paul McCartney: The Lyrics,” a massive two-volume box set of printed lyrics for “154 of his most meaningful songs.” Evidently “Wonderful Christmastime” is not meaningful, as it is not included in this set.

Nevertheless, due to its making much more sense, I had personally concluded that it is, in fact, “mood” (though I never seriously doubted it).

Then, earlier this week, the author of the witchcraft tweet followed up with a notice that @PaulMcCartney has since deleted its 2019 “moon” tweet, and that McCartney himself has addressed the controversy in a recent interview:

No, it’s ‘the mood’. And you know what, I’m thinking about Liverpool Christmas parties, that’s really all I’m doing with that song. “The mood is right, let’s raise a glass, the spirit’s up” – you know, all the stuff you do at Christmas.

Thanks, Paul.

Leave a Comment