Intelligence is the ability to adapt to change. These are the words uttered by the great Stephan Hawkins and can justly encapsulate the core meaning of the Agile concept which has revolutionized the way we conceptualize and produce software. The tech industry has gone through sweeping changes that have resulted in expeditious software production. Gone are the days where software teams would spend months and even years to deliver on new features to improve the experience of their customers. Now, changes can be implemented in a matter of weeks, if not days. A major push towards the change has been due to the maturing of cloud services and CI/CD practices.
Considering the cloud alone, serverless is the latest trend. A technological concept that if understood and implemented properly is one that can further our strive towards true Agile development. Therefore, the aim of this piece is to revisit the core concept of Agile, go through the developments in the tech industry and highlight the future of Agile with serverless as a practical base.
A Manifesto Worth the Revolution
Before we can dive into how Serverless can drive the Agile revolution, some misconceptions about Agile must be cleared. The first such misconception is that Agile is not a methodology or specific rule of development practice that must be followed. Instead, it is a set of values and principles, that when adopted can lead to better software development, and higher quality products being output.
The Agile manifesto is only 68-words long but thrives in its brevity. It lists a set of values and continues to declare that the believers in the manifesto give precedence to the values listed on the ‘left’ over those on the ‘right’.
So it should be understood that there is no tool or product that allows a company to take up Agile, but rather the set of values and principles itself. Any company wishing to be Agile should hardcode the values as per the manifesto and then explore the tools and technological practices out there to see which best fits their interests.
It goes without saying, however, that there are services out there in the market that do allow for companies to become Agile. It does not mean that any company that subscribes to these Agile services will become Agile in practice. Just because one company or team is using a certain service, does not mean that the service will work for your company or team. Constant introspection is required to note which practice or service is working for you at that specific time.
Now, all of this may seem contradictory to the motive of this piece promoting serverless as the way to further Agile endeavors. However, serverless itself is a concept built out of the benefits of Agile as will be apparent in the following sections and paragraphs. That is the second myth that I intend to bust. There is no one way to go serverless, but rather there exists an array of serverless tools and practices, allowing you the flexibility to make serverless work for your company or team while preserving the core characteristics of the concept that are in tandem with the values outlined in the Agile Manifesto.
From Years to Days
The time it takes to conceptualize an idea to finally pushing it out on the market is decreasing. Before, it used to take months if not years to release new features or products. This is simply because releasing software was expensive, and the infrastructure could not support software releases easily. So what changed?
Two monumental changes in the industry occurred that facilitated the reduction in lead times of software. These were the introduction of CI/CD tools and the maturing of the public cloud. Looking into public clouds more closely, one of the greatest benefits that we saw was the abstraction of underlying architecture allowing us to concentrate on business logic.
The fact that we no longer had to host on-premise servers, maintain our own databases, and worry about operational issues such as scalability and security meant that lead times now decreased. We no longer had to worry about releasing a feature or service as much as we could now focus on the value and quality of the product.
Ease in deploying software now meant that we could continuously deploy software, get customer feedback and then redeploy by acting on this customer feedback. Yes, we started following Agile patterns.
More specifically we began to value customer collaboration over contract negotiation. Contract negotiation was vital in product development before, because of just how long it would take and the cost incurred in building and deploying the product. Now, we could cheaply deploy new versions of software, ask if the customer likes it or if they would like to change something, and go back and deploy it again with the new changes. This also meant we could now respond to change over following a rigid plan.
The Illusion Within a Container
So far we saw the benefits that public clouds had on the general practice of software development. The push towards Agile thanks to the abstraction of architecture and cost-effective services provided by cloud vendors was a great leap in the industry, but not one without its hidden costs.
Yes, it has become more feasible to develop and release software, and we have seen a significant decrease in lead times. Nevertheless, as per the current status quo of developing on the cloud, running on containers one of the most popular options. This, however, adds an added complexity which is container orchestration. Therefore, lead times may have improved, but further progress is now being impeded by the complexity of containers.
For example, setting up an EC2 instance could involve setting up networking, security, scaling policies and much more. Such operational burdens are still sucking up valuable time that could be spent on developing business logic.
The Improvements with Serverless
The contribution that containers have had on the software development process cannot be brushed aside just because of a few operational costs. However, the solution to these operational costs in the form of serverless must not go unnoticed. We are now seeing incremental improvements to how software is being developed on the cloud through the lens of Agile development.
Firstly, it must be understood that serverless does not only mean AWS Lambda functions. In fact,a serverless is any service that displays the following characteristics:
- Pay-as-you-go model where you only pay for what you use
- Server management is abstracted to vendor
- Automatically scalable and highly available
These three characteristics of serverless allow complying services to fall under the Agile model. Serverless has made it so easy to deploy code, that now we can experience shorter lead times as compared to what we have at the moment with containers. Developers do not have to orchestrate new containers or scaling policies due to the changing needs of the customer. Hence we see a greater potential for customer value, or in other words, the values on the left side of the Agile manifesto.
Moreover, due to the pay-as-you-go characteristic, producing test software for customer feedback could be potentially cheaper. You no longer have to incur continuous costs for deploying test versions of your code for just a handful of customers. Thus making it easier to intertwine with the Agile values.
However, similar to the overhead costs of containers, there may be some hidden operational burdens that arise with serverless. One such cost could be the lack of monitoring or security. If there is a problem in either monitoring or security, then evaluating your software to make improvements can become difficult. This solution though is solved by the SaaS services in the community that is easing the operational burdens on a scale that is not fully seen when it comes to containers.
For example, Thundra.io has continuously strived to break down the operational burden of monitoring and their services bring full observability to Lambda functions. Hence allowing for developers using serverless services on AWS to continue with their Agile endeavors.
Another example of external SaaS services aiding the use of serverless is PureSec. Even though the vendors of serverless services assure basic security, the default level may not be enough to ensure the intended customer value. Therefore PureSec, recently acquired by Palo Alto Networks, is committed to providing serverless security to ensure that it does not hamper with the Agile process.
In conclusion, we see that as companies become more Agile thanks to maturing cloud services, we witness shorter lead times. These lead times could be further improved if the operational costs incurred by using cloud services such as containers, could be mitigated. The solution is serverless with its increased abstraction of the underlying architecture. Hence giving us that extra push in achieving true Agile practices and reaping the full benefits of it.