Ruslan Grumble - Fotolia

News Stay informed about the latest enterprise technology news and product updates.

Open source, adaptable infrastructure key to Etsy platform business model

How does Etsy keep its platform business model in tune? Deploying up to 60 code changes a day is par for the course.

This article can also be found in the Premium Editorial Download: CIO Decisions: Mastering the cloud contract:

Etsy Inc., the vast exchange for handcrafted and vintage items, relies on open source technology and a high-tempo software release schedule to support its platform business model.

The company, which in April completed its initial public offering, has become one of the most visible platform businesses, carving a niche in the arts-and-crafts field. On the supply side, some 1.4 million artisans and merchants have joined Etsy's worldwide community, selling everything from bed knobs to broomsticks and even a number of kitchen sinks. The platform currently offers 32 million items for sale. On the demand side, Etsy reports 1.4 million active buyers who generated sales of $1.93 billion in 2014.

The Etsy engineering team's top task: maintain a platform that keeps those constituencies happy.

John AllspawJohn Allspaw

"Engineering must ensure we do our part to maintain a platform that serves the needs of our community, first and foremost," said John Allspaw, Etsy's senior vice president of infrastructure and operations. "That means across the organization we're focused on creating the best buyer and seller experiences, handling everything from payments, trust in the marketplace, to search and data engineering."

For an emerging platform business model, information technology may not top the owner's agenda. Building a community and setting the ground rules for participation and conflict resolution are often the first priority. IT, however, tends to become a higher priority as a platform scales and matures. That's been the case for Etsy, which was founded in 2005. Today, Etsy's technology infrastructure plays a critical role in the current stage of the platform's evolution.

"We've developed a platform that is geared toward continuous deployment, speed, and long-term operability and performance," Allspaw explained.

Open source and the platform business model

Etsy is largely built on open source technology, according to Allspaw. At its core, the company's platform stack includes PHP and MySQL, Hadoop and Scalding, and Solr/Lucerne/ElasticSearch, he explained.

Engineering uses PHP, a scripting language, to write most of the platform's server-side code, while the MySQL database stores the platform's data. Hadoop, a programming framework that supports distributed processing, and Scalding, a domain-specific language, drive the platform's analytics capabilities. Solr/Lucene/ElasticSearch, meanwhile, provide the platform's search capabilities. In addition, Etsy uses Apache Kafka, a distributed messaging system, to manage its data pipeline, Allspaw explained. Apache, said Kafka, can "handle hundreds of megabytes of reads and writes per second from thousands of clients."

John Hagel III, co-chairman of Deloitte LLP's Center for the Edge and author of the recent report, The Power of Platforms, said the open source approach is making inroads among younger platforms.

John Hagel IIIJohn Hagel III

"The newer generation of platform companies follows this pattern quite a bit," he said. "When you think about it in economic terms, it is dramatically lowering the cost of entry to set up a platform."

Hagel said the earlier platform companies -- those emerging in the 1990s, for example -- often developed everything from scratch, either out of necessity or preference. He noted that some companies convinced themselves that they had to do it all on their own to create differentiated functionality. But he said more companies now realize that while they may want to include some propriety functionality on their platforms, most of the supporting technology can be provided via open source.

Open source, versus do it yourself, also shortens time to market, which is critical for a platform business model, Hagel added.

"Getting to critical mass quickly is a key business imperative if you want to be a platform business," he said. "The open source option significantly compresses that lead time."

Toward an adaptable infrastructure

Flexibility, along with speed, is another important aspect of Etsy's platform infrastructure. The engineering team puts a premium on the ability to make rapid adjustments to its technology stack. To do that, it focuses on a limited set of tools.

"Our overarching philosophy is that of 'Code as Craft,' [that is], investing in and mastering a small set of well-known tools and working within our systems," Allspaw said. (The company's engineering blog goes by the name of Code as Craft.) "This level of familiarity, expertise and access grants a huge level of confidence, so we can focus on the tasks at hand."

The approach makes it easier to fix problems in the infrastructure when they arise.

"When something breaks, we want to be able to get under the hood in order to devise solutions quickly, saving us time and resources," Allspaw noted. "Our goal is for our code and infrastructure to be adaptable, so we make small changes frequently."

Indeed, Etsy sometimes deploys code up to 60 times per day. The frequent updates ensure that "we're nimble, and out stack more reliable and scalable."

Hagel added that frequent iteration also helps platforms accommodate community needs. He said platform owners should watch and learn from the platform's participants. Which features do they use? What are their unmet needs? Hagel said the most successful platform companies are the ones that rapidly evolve as they see what's needed.

That rapid evolution, ideally, begins from a baseline feature set. Hagel described this core functionality as the "minimum viable platform." This approach borrows from the concept of the minimum viable product, which holds that a product should launch with a basic feature set and then expand based on customer feedback.

"Just focus on that minimum functionality at the outset and start building that critical mass and evolving very rapidly over time," Hagel said.

He said platform managers should apply that thinking to a platform's governance as well as its technology. He advised platforms to start out with a limited set of rules for participation.

"Don't over engineer the governance structure or the standards and protocols," he suggested.

The Etsy Way

Holistic: Etsy builds, maintains and operates the platform by using holistic teams; collaboration is an essential part of the culture.

Integrated: Etsy's engineers, product managers and designers are integrated

Aligned: Business and technology intermingles even at the leadership level: Chad Dickerson, Etsy's CEO, was previously the company's CTO.

Open to learning: Since technology is always in flux, Etsy is continuously "learning, iterating, measuring and experimenting," company SVP John Allspaw noted.

Etsy's platform outlook

Platforms are outwardly focused by their nature. Etsy's engineering team follows that cue with its open source orientation. As the team leverages open source technology, it also contributes software to the open source community.

"We regularly contribute a large amount back," Allspaw said, noting that Etsy's open source projects take up four pages on the GitHub developer community's website.

Allspaw said one of Etsy's most popular open source contributions is statsd, a JavaScript framework.

In addition, Etsy built a machine learning framework, Conjecture, which it has made available as open source software on GitHub. Conjecture lets developers create machine learning models in Hadoop using the Scalding language, according to an Etsy blog. In one example, Etsy employs predictive machine learning models to "estimate click rates of items so that we can present high quality and relevant items to potential buyers on the site," the blog stated.

But as Etsy looks outward, it's encountering competitors as well as collaborators. Amazon.com, which helped pioneer the platform business model in the 1990s, recently launched its Handmade store, which is poised to make a run at Etsy. The Etsy platform, however, has room to differentiate as other rivals move in on its arts-and-crafts turf.

Hagel categorized Etsy as primarily an aggregation platform, specifically a marketplace and broker platform that connects vendors making items to consumers within highly targeted niches. He said Etsy-like platforms have an opportunity to evolve into a learning platform, in which participants not only buy and sell goods, but learn from each other. Entrepreneurial artisans active on the platform may have no experience in building a home-based business, but the platform could provide an environment where sellers can connect and share experiences to accelerate learning, Hagel noted. On the buyer side of the platform, pottery fanciers, for example, could compare notes on how to build a collection, he added.

"Once you've built out those learning environments and get active participants, it becomes harder for a … company to come into this space to challenge a platform," Hagel said.

Next Steps

Read more SearchCIO coverage on platform businesses:

The CIO role in designing and maintaining platform business models

Five big challenges for platform businesses

MIT Platform Strategy Summit: Here comes the platform revolution

McCormick spinoff Vivanda leverages a platform business model to spice up food sales

This was last published in July 2015

Dig Deeper on Digital business management

PRO+

Content

Find more PRO+ content and other member only offers, here.

Join the conversation

2 comments

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.

Flexibility is an important aspect of the Etsy platform. Has your IT engineering team taken steps to make the company's customer-facing infrastructure more adaptable to change?
Cancel
I like that they've embraced open source tools, both leveraging them and contributing as well. At my organization there is a lot of red tape around using or contributing to open source code; I guess it makes the legal team nervous. 

Sounds like Etsy is doing something right. 
Cancel

-ADS BY GOOGLE

SearchCompliance

SearchHealthIT

SearchCloudComputing

SearchMobileComputing

SearchDataCenter

Close