Agile Development for Science

Scientific work can also benefit from principles derived from software development

Managing research projects and teams can benefit from the methodologies and processes developed in other fields of management. In recent decades, software development became a major industry that needed to establish its own set of rules and procedures. People working in the software industry and young scientists such as Ph.D. candidates and postdocs share a lot of qualities and interests. Therefore, it can be useful to check what the software industry has developed in terms of management and see what procedures can be extrapolated to the scientific environment.

The Agile software development describes a particular approach to software development methodology based on iterative development. Its core value is that requirements and solutions evolve through a collaborative process between developers and customers. The principles of the Agile methodology are summarized in four statements in its manifesto. They are:

  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

Agile in itself doesn't establish any particulars regarding programming but just how to develop a workflow and a working environment. In the last few years, Agile has attracted a lot of attention, extending its concepts away from software development, and moving into project management. Even though the core principles were drawn from a programming context, scientific projects could also benefit from the Agile ideas, but with some reservations.

Some of the main points of an Agile process, such as responding to change over following a plan are a routine reality in a lab. Plans are hard to follow because a researcher cannot be sure of what lays ahead. More interesting questions arise or some things just don't work as expected. Being flexible to overcome obstacles and to change focus when more important questions can be answered, is a needed attribute of successful researchers.

Collaboration over negotiation is also common in a scientific setting, where a lot of agreements are done verbally, without almost any negotiation. Sometimes it is not even an agreement, but just mere curiosity that sparkles a collaboration. Collaboration in science often starts with complementary capacities, for example, two different groups which developed expertise in different techniques. Negotiations come at a much later stage, only when the collaboration was successful and researchers need to agree on how to share the credit.

The other two principles of the Agile methodology, however, are worth discussing in detail in the context of scientific work. To value individuals and interactions is a general principle to which few researchers will oppose. It is better to allow creativity than to be fixated on a specific process or tool. Moreover, great ideas may appear from a discussion or an interaction with another researcher. However, it is also important to value the processes and tools that each researcher has available in the lab. Even though questions can appear in any setting, only places with the proper setups will be able to answer them. The opposite is also valid, posing questions to which one has the proper tools to answer.

The context of every research is different and every field is particular; valuing individuals is part of a healthy working environment, but one should not lose sight of the established processes. Processes can guarantee the quality of data and the moral integrity of researchers. It doesn't mean things cannot be changed for the better, but there should be a limit to how much an individual can be valued over an established process. A brilliant scientist will stop contributing to the group if he or she doesn't document the work or doesn't follow the procedures established to share resources, for example.

Working software over extensive documentation should be better rephrased as Results over extensive documentation. In this particular point, an Agile process can be the opposite of what a lab manager wishes, but it is exactly what commonly happens. Reproducing results is only possible when extensive documentation is available, allowing others to perform the exact same measurement. The pace at which science is moving, however, values quick results over thorough research, allowing little or no time to provide extensive documentation about experiments and results.

Documentation in science has a different value than for software development, but the rhythm at which both evolve is fundamentally the same. Science is based on building on each other's work, while software development can be done to overcome a very specific obstacle. The difference between a scientific work and software development is, therefore, the life cycle of the results. While one expects to build experiments on top of the knowledge generated in a lab, he may not use an available program as a starting point for further development.

The Agile manifesto provides not only the four values stated at the beginning but also twelve principles to follow. Some of which are very relevant to a lab manager. For example:

  • Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
  • The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
  • Continuous attention to technical excellence and good design enhances agility.
  • Simplicity --the art of maximizing the amount of work not done-- is essential.
  • At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

The Agile manifesto cannot be extrapolated directly to scientific work but nonetheless, it can provide a good source of inspiration. It may be time to develop a manifesto for research, that can be used as a guide for lab managers seeking advice on how to improve the working conditions at their labs.

Header photo by Giu Vicente on Unsplash

The article Agile Development for Science was written by Aquiles Carattino.
Support us
If you like the content of this website, consider buying a copy of the book Python For The Lab

Other interesting articles for you

The MVC Pattern for Lab Projects

When you develop applications for the lab, probably you want to be able to re-use parts of the code and allow other scientists to build on your work. This problem has been discussed at length around other applications, such as building web services. And in a certain way, developing a …

read more

Should Labs Move to Open Source?

We can agree that researchers exchange published papers as a currency, and its value is the number of citations each has. Accessing conferences, faculty positions and grants is to a great extent achieved on the basis of researchers' success history. However, publishing software is not counted towards a scientist's productivity …

read more

© Copyright 2020 Uetke | Contact Us | RSS feed | Privacy Policy | Cookies