Turbo Charge Agile Processes with Deep Learning

The key to leveraging Deep Learning, or more broadly AI, in the workplace is to understand where it fits within an agile development environment.
By Carlos Perez, Intuition Machine.

Image credit.

There is an inordinate amount of contribution from the software development field to the way we do work and the way we run businesses.

In the early 2000’s the “Agile Manifesto” was created in response to our collective inability to effectively manage complex software development. The tenets of the manifesto are as follows:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Software development is fundamentally a knowledge creation activity. The factory floor processes that are so effective in the physical world simply don’t apply in the virtual world. The agile manifesto was created to challenge our conventionally wisdom of processes that was prevalent at that time.
Agile development has evolved since then and what we have seen is practice of employing automation where needed. The key advantage of agile is its ability to respond quickly to change. If you can automate sections of that feedback loop without compromising, then you have a tighter loop and therefore a more nimble process.

The concepts of “continuous integration” and “devops” are enabled because of automation. In other words, agile processes are effective because there is a clear understanding of where automation can be best leveraged to enhance the human workflow. The correct prescription for introducing automation in one’s processes is to begin with a process that is centered around humans and then to augment that with automation.

If this is done in the other direction. That is beginning with a factory floor model (i.e. waterfall) and then automating the parts, then you are left with an even more inflexible processes and a lot of unhappy workers. The mechanization of work may have worked in an earlier era that lacked the coordination and communication capabilities of computers. In a world were computation and devices are abundant and pervasive, the only efficient workflow model that one should consider is that one based on adaptive principles.

Ideas coming from the agile development have lead to further usage in the world of startups. The startup world is a world were agility is a critical capability for survival. Startups are buying into the agile philosophy of prioritizing feedback with potential customers. The purpose is to create a learning organization by effectively iterating through many business plans. For startups with a lack of vision, this kind of “The Lean Startup” process is better than having no process at all. How can we learn more quickly what works, and discard what doesn’t? That’s the essence of the approach.

The key to leveraging Deep Learning, or more broadly AI, in the workplace is to understand where it fits within an agile development environment. (It is of course obvious that Deep Learning development and operations should work within a continuous integration and devops environment. We will discuss this issue in another post.) The application of Deep Learning to one’s own processes requires a keen understanding of our human workflow processes and seeking opportunities to enhance, not a rigid process but one that is adaptive, through automation. The very notion of a complete replacement of a human worker through automation only makes sense when the original business process has always been mechanized.

What are the strengths of DL relative to classical AI technologies? The key strength of DL is that is that it is able to function in an almost biological and adaptive manner. That plays very well in the application of DL as a conduit to interaction with humans. We already see this DL applications in speech recognition and in gesture understanding. DL is essentially the new UI. A UI that is ambient, allowing its users to easily summon its capabilities for the task at hand.

Most business perceive DL or more broadly Machine Learning or AI solely as a prediction tool. That is a tool employed by
Image credit.

This is an example of a generative application. We can use DL to exhaustive explore a design space to serve as a “brainstorm” of ideas. This mode also works in the realm of planning and execution. DL can quickly explore many scenarios and present workers with a Chinese menu of promising options. We’ve written previously about some interesting examples of generative design: “The Alien Style of Deep Learning Generative Design”. We have only seen the tip of the iceberg here, there is so much more to explore in this space. One application of DL in the space of “Game Theoretic” design is enough to scare the bejeezus out of anyone.

To begin your discussion on how Deep Learning can help enhance your business, we will be glad to start the conversation at Intuition Machine.

Bio: Carlos Perez is a software developer presently writing a book on "Design Patterns for Deep Learning". This is where he sources his ideas for his blog posts.

Original. Reposted with permission.


  • Why Deep Learning is Radically Different From Machine Learning
  • Deep Learning Can be Applied to Natural Language Processing
  • Game Theory Reveals the Future of Deep Learning