Content, Creators, and Algorithms
How The Creator Economy Works, The Major Issues, and My Attempt to Fix Things
Let’s talk about content, specifically from the standpoint of creators.
How much of their time is spent creating content vs everything else?
50%? 25%?
Here’s the reality for most creators: it’s about way more than the content.
Why is this?
There are a number of factors that contribute to this, but I think there are 2 main factors that overshadow everything else:
Monetization
Discovery
Those two factors are the reason they have to spend well over 50% of their time on things that aren’t their content. Once they hit a certain level, they can focus on hiring experts to handle the pieces that they don’t want to deal with so they can focus on more than their content.
But early on? It’s all on the creator. So they have to learn all the pieces to enable those 2 factors, and unless their background is in those areas, they are probably going to suck at it because they’ve never tried before.
How do I know? Because I’ve been there. (Spoiler, I’m still there)
Discovery
It’s incredibly tough to be discovered by a large enough audience that monetization becomes relatively easy or to build a strong enough brand/product that you don’t require a big audience to get enough income to make a living.
So creators look for shortcuts.
What’s the biggest shortcut to audience growth?
Depends on your perspective, but it usually relates to two areas:
Search Engine Optimization (SEO)
Social Media Platforms
Let’s take a look at each in detail:
SEO
What is SEO? In a nutshell, it’s the process of helping Google (or other search engines) the ability to direct people to your site based on what people search for.
And it’s also the process of convincing Google that your site is good enough to rank for different search terms.
In order to do well, you need to understand 2 different pieces:
What are people searching for? If you don’t understand what people are typing into search engines, it makes it hard for you to give them the things they are looking for.
How does the search engine decide what content is “good”? If you don’t know what good looks like, it’s hard to make your content fit that definition so it makes the cut.
Notice what’s not here? The things you want to create content about. Those don’t really factor in nearly as much as the other two things.
This makes a lot of sense if you are a brand creating content, but less so if you are an individual creator. So search engines probably won’t be very useful to individual creators.
Social Media Platforms
I think most people have a pretty good idea on social media platforms and what problems they have, but I want to focus on one question:
What do social media algorithms try to do?
Spread the best content?
No.
They try to make the social media platforms more valuable.
How they do this depends on the type of social media platform, but the most common social media platform model is attention-based. They sell ads, so they need more people to stay on the site longer, because that leads to revenue growth.
This creates incentives that are at odds with creators, doesn’t it?
There’s a reason that social media platforms deprioritize external links, for example.
External links take people off their platform, so that’s not a good benefit to the social media platform. Therefore they won’t boost those posts.
So on Twitter, people write threads, or use screenshots, or things like that in order to keep people on the platform in hopes that their content will get spread a bit wider. But of course, these don’t fit well into the workflow of creators, who might have blogs, podcasts, video channels, newsletters, etc.
That ends up having an interesting effect. The goal becomes to maximize the number of followers on social media, because over time, those followers will hopefully discover the content the creator makes and end up in their “captured audience”, or the people who the creator can control contact with outside of the social media platform.
So creators end up trying to optimize all sorts of things on their social media profiles to make that work as smooth as possible.
Which helps the social media platforms and keeps them large and valuable.
But how much does it really help the creator? Of all the followers they get, how many end up in their captured audience? And how many end up buying their products/services?
Monetization
Now let’s talk about the other hard part: making money.
How do creators monetize their audience?
There are a bunch of ways to do so, depending on the size of your audience. Let’s run through some of the options, from smallest audience size needed to largest:
Services - High ticket offerings, tied to time, limited scale.
Products - Low-to-high price offerings, depending on the product. This is something that can scale. “Build once, sell twice”
Affiliate sales - Sell products for other people and get a cut of the sale. Depending on the product and how much the sale/cut is, can potentially be lucrative
Sponsorships - Usually requires that you have an existing audience to prove the value of the deal. If the audience is known and a good fit, can potentially be decently lucrative for a small-ish audience
Sponsored content - Smaller deals, usually a 1-off or potentially recurring deal where you can promote a product or service for someone to your audience.
Advertising - You get paid based on the number of views typically.
One thing that is evident from this list: the revenue comes from more and more indirect sources the further down you go. It goes from high level of effort for the creator and highest value to lowest effort for the creator, but limited value, leading to the need for highest number of views.
And yet, to hit those larger numbers, the creators do need to put in a lot of effort: effort understanding the algorithms, understanding their audience, and figuring out what content works the best, right?
It just so happens that the majority of the value created by that effort gets captured by the platforms providing the additional products or services advertised or that implement the distribution algorithm.
WTF Does This Have To Do With Anything?
This is definitely not the usual SaaS Factory issue. I’ll get into some of the stuff I’ve been working on soon, but there are a few ideas driving me forward in almost everything I do.
Content Creation and Distribution should be easier. I hate playing the algorithm games and in fact, I refuse to do so. While I understand SEO and how these algorithms want me to behave, I refuse to do so, because I don’t have the time to conform. I’d rather just share the things I’m thinking about in the format that I want to use, and that will be that. I’d much rather spend that extra time building things that are interesting or hell, just living life. So I’ve been trying to figure out how to get the results without playing the default game.
The value of content should be based more on the level of thought that goes into it than the number of eyeballs it gets on it. I’ve had a bit of fascination with Medium and the way they pay writers based on amount of time spent on articles as a percentage of subscriber revenue, but even that has devolved into the same thing that most content platforms have.
Competition between content is done wrong. In theory, competition should surface the best content, right? Over time, the better content will win out. And yet, that’s not true of most platforms. Here’s why: the content is forced to compete on the wrong attributes. Think about YouTube. When you search for something, you then get a list of videos. And you have to choose the video based on two things primarily: the title and the thumbnail. When you want to win a battle of title and thumbnail, there’s something that always wins: the highest emotion provoked, whether positive or negative. Pay attention to the titles of the videos you pick to watch from a search. How many are trying to show the highest possible level of emotion? How many thumbnails are exaggerated physical reactions? When competing on a small number of criteria, extreme always wins.
As I’ve been experimenting with a number of the projects I’ve built, I’ve spent a lot of time experimenting with content discovery for these reasons. I want to build something that helps the real creators succeed. I want to help the creators who are sharing their earned insights from their life, not those who have been focused on all the tricks to grow faster. I want people to compete where it matters: the ideas contained within the content, not the structure of the content.
I wrote about the next evolution of digital creators who will rise up.
And that brings me to my current build: a new type of content distribution algorithm.
Instead of worrying about what platform someone is one and optimizing for discovery on that platform, or worrying about what someone might be looking for, I’m experimenting with just-in-time content discovery.
What if you could get the content you need before you even know you need it?
Now, there’s a level of caution I have to use here, because I don’t want to hijack anyone’s data or give people a new way to interject their content to capture your attention when you don’t want to give it to them. And to do that, I need to give people the ability to customize the algorithms they are using to deliver themselves the content they need.
And the good news is that the newest large language models can help me do that.
In the last issue, I mentioned my build of 5 Minute Show Notes. It ended up not amounting to much, people weren’t that interested in it to a level that it would be a profitable way to spend my effort, but it did lay some foundations for inputting podcast data.
So I adapted it.
And I had Who Followed Me? still in testing as well, which offered me some additional functionality I could add in.
That combination of functionality showed me that I could do something kinda interesting. I could take a look at the topics of recent Twitter posts and I could create semantic embeddings of podcast transcripts.
Then I discovered Pinecone, which is a vector database that allows me to easily search vector similarity, which means that I could match recent tweet topics to podcast topics and see where the best fit would be.
I tested out the initial version of it live on Twitter with my own podcast episodes:
I’ve heard back from 1 episode I recommended and it came back quite positive and more importantly, I got some volunteers who were interested in the idea of it. So now, I’m planning on reaching out to some content creators who might be interested in developing their own algorithms. I’m planning on offering three types of algorithms that users can create and customize: creator (distribute your content), curator (distribute the content of others), and consumer (find the content you like/need). I think there’s a lot of potential here.
But I’ve got to prove it and I think the first step is to focus on the creators, specifically podcast creators. There’s a known issue with podcast discoverability and I think this could be a solution. I have a shortlist of creators that I’m going to reach out to and gauge interest and go from there.
Experimentation Framework
Let’s talk some nuts and bolts now. When I decided I wanted to pivot away from the whole 5 Minute Show Notes thing, I wanted to do so as easily as possible. On the backend, I ended up adding in the API calls to Pinecone to store the embeddings and tweaked my data models a bit.
Because of the length of content (my podcasts are typically ~1 hour), I had to split the transcripts into chunks. So I’ve got a model that references the content and a model that references specific chunks, rolling up into the parent model. Then each chunk has a GUID that I use as an identifier that is stored with the embedding in Pinecone, so I can reference it. Then, because I can match multiple chunks in a query, I have the search return the content with the most chunks matched in the 10 highest matches.
I’ll be tweaking that a bit to see how well it works for different versions of that algorithm, but it seems good enough for now.
For the experimentation though, I wanted something quick and easy. I didn’t want people to have to sign up to anything for me to make a recommendation, so I created an Airtable table that just has 2 fields: Twitter Username and Created. Created is the timestamp that the row was created, and is important because N8N uses that to determine what rows have been added since the last run. Then I threw on an endpoint that allowed me to pass in the username, hooked up the analysis flow, ran the query against Pinecone, picked the best match, and emailed myself the result. All said, only took about 1 hour to set up, so pretty happy with that!
On the content entry side, I ended up doing something similar. I created a table in Airtable that had all of the fields I cared about: title, link, summary, fulltext, etc for each podcast episode. Then I created a form in Airtable that allowed me to put in all that data easily. I set up a trigger like the other one and an endpoint, and that kicked off easily. So now I have easy ways to analyze accounts based on Twitter accounts and easy ways to add content to the database and create/store the embeddings. That means I can move very quickly now as I experiment with things.
Extra Considerations
One thing I do have to keep in mind is my costs. It’s not free to run embeddings, although with the newest model that OpenAI released, it is a lot cheaper than it was previously, but still non-zero. And with Pinecone, I’m on the free trial plan, which has limits that seem ok for now, but not sure when I blow past them, and that’s a cost. So I’m doing some analysis on what I’ll have to charge. I’m leaning toward two tiers of service: a very basic plan that will be in the $25-50/month range and will be mostly initial setup and very specific features and a more white-glove type approach to algorithm development, where I’ll help them learn more about what works and offer customization options to their content recommendations.
Once I get calls set up with creators, I’ll be able to test the waters and see which one is more attractive to them. My guess is that I’ll be able to get them on the lower tier fairly easily, at least to try it out, and then if it does extremely well, they might be interested in the premium offering.
But time will tell.
Really thoughtful piece, Leo! Excited to hear more about how you're building this.
I also appreciated our call on AI. It's not fully automated yet, but we're building out http://freshprompts.com, which is based on our convo.