DevOps transformed software development. By bringing development and operations teams together, companies achieved a dramatic boost in both speed (agility) and efficiency. But DevOps is just the first chapter. The future of software development lies in platforms. These platforms are like building blocks–scalable and reusable, enabling companies to develop and deliver software even faster and more efficiently than ever before. The best way to achieve this is by making developers happy and more productive.
Why we need platform engineering
Development and Ops teams are navigating a lot of obstacles these days. Not only are tools, frameworks, languages, data storage, and messaging systems becoming increasingly complex, but the shift to cloud environments and closer Dev/Ops integration adds another layer.
While this can be exciting for small teams at first, it can quickly become overwhelming in daily work. The sheer amount of tasks and decisions can surpass what's manageable, leading to a significant slowdown in actual development. Terms like "cognitive load" and "burnout" come up for a reason. Development and Ops teams must balance mastering complex technologies in development and production while also integrating them securely and in accordance with company standards.
So, how do we break this Dev/Ops cycle without stifling developer freedom? Platform engineering is the answer. It's a software engineering discipline that provides developers with a foundation: infrastructure, tools, and established processes. This helps them be more effective and reduces repetitive tasks.
A brief history of platform engineering
The concept of platform engineering emerged a few years ago. Tech giants like Netflix and Amazon pioneered self-service developer portals for tasks like provisioning virtual machines. In 2018, Martin Fowler's article on digital platforms identified five key areas for successful transformation: infrastructure, interfaces and frameworks, self-service, monitoring, and customer-facing technologies.
Another important influence came in 2019 with Team Topologies by Skelton and Pais, which introduced the concept of platform teams and their role within the development organization. The book also highlighted negative DevOps patterns, where unclear responsibilities hindered progress. These ideas fueled the rise of platform engineering in the following years.
You can't buy platform engineering
Hold on! Platform engineering isn't a product you can buy. It's a way to maximize the benefits of DevOps. The core idea is for a dedicated platform team to build a technical platform specifically designed for your software developers' needs.
This platform is often called a "Platform as a Product," emphasizing that it's not a static piece of software. Instead, it's a constantly evolving solution tailored to your company's developers. Here are the key principles behind it:
- Make developer lives easier. Automate repetitive tasks and simplify complex processes.
- Offer choices, not limitations. Don't force developers to use specific tools. Provide options that hide the complexity of underlying technologies.
- Self-service is king. The platform team treats the platform like a product with features developers can access on their own, like a buffet.
Platforms for cloud native application development
Imagine a cloud-native computing platform as a one-stop shop for all your cloud development needs. It brings together various tools and services, designed specifically for how your team works. This central layer makes it easy to find and use the features you need, no matter what kind of application you're building. A great platform offers a consistent experience, like web portals, project templates, and self-service options, so you can manage everything in one place.
The features a platform offers should be based on what users and stakeholders need and curated as a product by assessing their needs. While platforms provide these functionalities, the platform engineers (also known as the platform team) don't always have to build everything from scratch. Specialized external services or dedicated internal teams can handle the underlying technical aspects. The platform itself acts as a central layer, ensuring everything works together smoothly and meets your organization's specific requirements.
Platforms act as a bridge between the underlying technical providers (think cloud services or internal tools) and the people who use them (like application developers). In this role, they not only provide access to these capabilities but also enforce best practices. This ensures capabilities like security, performance, cost-effectiveness, and a consistent user experience for everyone. Figure 1 illustrates this relationship between products, platforms, and capability providers.
In this overview article, I’ll only focus on the two big areas: platform interfaces and platform capabilities.
Platform interfaces
The “platform interface” layer summarizes the primary “interfaces” that developers can use when interacting with the platform. The choice of interface is left to the developers. The “interface” here often includes development environments (IDEs), browsers, or portals (like Backstage from the Cloud Native Computing Foundation), but of course also command-line tools (CLI) or version control systems (GitHub, etc.).
A fundamental idea here is to leave the well-known and established development process with existing tools untouched as much as possible. It is also essential to give developers the freedom to choose their favorite tools. This category also includes tools for managing and working with containers, such as Podman Desktop.
Platform capabilities
Think of a platform capability like a toolbox. It holds different features, which are like the individual tools. For instance, "observability" is a capability, and features within that toolbox might be collecting performance data (metrics), tracking how data flows (traces), and monitoring logs. You might even have tools to keep an eye on costs and energy use. The key is to decide which features within each capability are most important for your organization. We’ll dive deeper into the platform capabilities and show you more examples in a future article.
Golden paths: Software templates as a shortcut for developers
Internal developer platforms often use "golden paths" or "software templates" to streamline development. While Spotify coined the term "golden path," other companies use different names like "paved road" (Netflix). Regardless of the name, this concept offers a simplified way to build or modify software components.
Imagine these templates as form-driven generators for specific application parts, like services. Teams that can stay on this "path" leverage prebuilt configurations and best practices, allowing them to get things into production faster without needing to be experts in the underlying technology. The more powerful these templates become, the more developers use them, leading to consistent applications across the company.
What’s included in a software template
A basic template typically includes:
- Source code starting point: This could be a simple "Hello World" application preconfigured with essential resources to get any developer started quickly.
- Deployment pipeline: This automated pipeline builds the code, prepares the artifacts, and potentially deploys them to production. The pipeline may include security checks or approval steps depending on company practices.
- Configuration files: These files configure the application itself, often using tools like Helm charts or Kustomize in the Kubernetes world.
- Monitoring integration: Depending on needs, these templates can include preconfigured monitoring for performance, errors, and other aspects. Ideally, what gets monitored is defined declaratively in a separate file for easy management.
Software templates are often configurable, allowing for some customization. The platform's ability to handle these variations is crucial for its adoption. This allows for changes to pipelines, backups, test data updates, etc., while still maintaining a balance between ease of use and handling special cases. The platform team plays a key role in deciding when something becomes a template and when it remains a custom configuration.
While templates offer a faster and more consistent path, they aren't the only way to integrate code. There should always be options for experimentation and handling unique situations. Ideally, the templates are powerful enough to handle most cases, but there should still be ways for developers to dive deeper if needed. This transparency is especially important in "shift-left" cultures where development teams are also responsible for operations.
Tips for building a winning platform engineering team
Platform teams act as a strategic force that provides developers with the foundational building blocks they need to be more productive. Imagine them as the architects who design and build the infrastructure upon which developers construct applications.
There are key ingredients for a successful platform engineering team. Here are a few:
- Product focus: One of key success elements to adopting platform engineering is to embrace a platform-as-a-product mindset.
- Clear vision: What's the goal of your platform? Is it to boost developer productivity, speed up releases, save money, or improve software quality/security? These goals should be measurable, and many companies use the DORA metrics to track progress.
- Communication: Platform teams need to collaborate closely with developers, DevOps teams, and product teams. Open and honest communication avoids misunderstandings and helps everyone work towards the same goals. The company culture also plays a big role–a supportive environment fuels success.
- Automation and standardization: These are top priorities for platform teams. Identifying and automating common use cases and functionalities is crucial for making software templates and the platform itself more widely adopted. A broad understanding of existing projects helps with this automation.
- Continuous learning: Platform engineering is a young and rapidly evolving field. The best teams are always learning about new trends and technologies to stay ahead of the curve.
Developers: Building the platform, brick by brick
Platform engineering thrives on collaboration. While platform teams lay the foundation, developers play a crucial role in shaping and strengthening the platform. Developers are essential ingredients to any platform engineering journey. Their everyday experience in delivering software solutions exposes them to frequent pain points and repetitive tasks. By sharing these challenges with the platform engineers, developers make it possible to pinpoint areas where automation or prebuilt components can significantly streamline the development process.
Address specific developer needs
Real-world scenarios are essential for building a platform with practical value. Developers can contribute valuable insights by providing detailed use cases that showcase typical development workflows. This gives the platform team the opportunity to tailor components and templates to address specific developer needs, ensuring the platform directly tackles real-world problems.
Drive continuous improvement
Once platform features like templates and automation scripts are rolled out, developers become the primary testers. Their feedback is crucial for ongoing refinement and improvement. By providing clear and actionable feedback, developers empower the platform team to continuously optimize the tools and resources they offer, ensuring the platform remains relevant and effective.
Promote code consistency
But developers aren't just testers; they're also builders. They can directly contribute to the platform by creating reusable components–well-tested functions, libraries, or code modules that address common development tasks. Sharing these components with the platform team makes them available to other developers, promoting code reuse and consistency across projects.
Equip teams with knowledge and best practices
Finally, the knowledge of experienced developers is mostly undocumented treasure. By documenting best practices and coding standards, developers can help embed these practices into the platform itself. This can be achieved through code comments, tutorials, or internal knowledge-sharing sessions. In essence, developers play a critical role in capturing and disseminating valuable development knowledge, ensuring it becomes an integral part of the platform and benefits everyone.
Learn more about developer portals
If you want to learn more about platform engineering and developer portals, make sure to download the free and brand new e-book Developer Portals: Prepare to Perform with Red Hat Developer Hub. Red Hat Developer Hub provides a unified portal and standardized software templates to simplify and enhance the development process. Visit the Red Hat Developer Hub product page to learn more.
