How are engineers creating professional software tools?

This article was written in collaboration with Peter Konijnenbelt (Arcadis), Johan Tuls (BAM), Kelvin Roovers (BESIX), Tjeerd Hermsen (Sweco), Nynke ter Heide (Vidabo) and Kevin van Giessen (VIKTOR). We have something in common: we create software tools that have a big impact in our organisations and are used by many people. However, none of us have a background in software engineering. We are trained engineers in our different disciplines. We believe you can do this too. You just need to choose the right tools and follow a good method. Here we share our experience.

The start of a series

This is the part of a series of articles about engineering and Python. Each time we will go deeper into engineering content and talk about solving specific problems.

In this article we will talk about:

• The digital transformation and why engineers are programming
• The four components of professional engineering tool: Automation, integration, optimisation, and distribution… and why you should consider all of them when creating your tools.
• Why we recommend using Python for making your own tools and our experience with other languages or software.
• Creating digital building blocks and their impact on the future of engineering
• How to make your code accessible for everyone, even to people that cannot code!
• Why and how engineers are using VIKTOR to create software tools that are accelerating the digital transformation of their companies.
• Tips & tricks you can apply to your own projects

The digital transformation

Digitization is growing exponentially thanks to the increasing accessibility to user-friendly software development tools. Gartner defines this movement as the Democratization of Technology. This means providing people with easy access to technical expertise without extensive (and expensive) training. This is often referred to as “citizen access”.

This changes the way software is developed. In construction and engineering companies, for example, there is a transformation. Software is no longer something for the IT department. Engineers are empowered to make their own software applications. Management sees the value of such powerful tools and creating them has become a company policy. Like this example from BAM, to design subway crossing under roads or railways.

Why are engineers programming?

In engineering, every project is unique, at least that is what most engineers will say. This means you cannot copy one design directly to another project; however, the design techniques will be applied in the same way in each project. For example, a bridge must always comply with same design code, and its beams, foundation and paving are calculated in the same way. In addition, finding the optimal solution requires many calculations. A time-consuming process when done manually and, perhaps worst of all, no fun at all.

Automating these types of repetitive processes can help to quickly find the best solution and make engineering work more fun. This is why most engineers start programming. After all, we did not study engineering to spend our days updating models and copying/pasting results into reports. We want to engineer awesome things.

“If you don’t learn to automate the repetitive part of your work, someone else will do it for you. You must learn this to stay relevant.” - Kevin van Giessen, VIKTOR

The four components of a professional engineering tool

I notice that each professional engineering tool created to solve a specific problem, for instance designing a bridge, has four distinguishable components: integrate, automate, optimise, and distribute.

Automation

By automation I mean that the program can perform a task automatically, such as generating a report or copying data from one place to another.

Integration

Integration is the automatic combining of different sources of information, which could be models, databases, Excel sheets or external software packages.

Optimisation

Optimisation is when the software tool calculates different designs or scenarios and helps you find the best solution.

Distribution

Distribution makes the software tools available to other. It transforms the tools you use into something anyone can use. As some of us have learned the hard way, processes, software packages and programming languages that seems perfectly suitable for one part become the bottleneck for the other. Distribution is the most underestimated part. For some reason, people often think this is something we can solve later, and that integration and optimisation are much more complex. This is not true; I have seen so many great projects end up in the drawer because they did not think of distribution and usability beforehand. That is why it is important that you consider all components and from the start choose the right tools.

Which programming language should I use?

Having the computer help you solve engineering problems is not a new concept. Many engineers have their own Excel or Mathcad sheets and VBA or Matlab codes. But there are some drawbacks to those programs, such as the difficulty to do version control, the lack of a user-friendly interface and codes that are hard to understand if you have not written them. Fortunately, a new language is rising in the engineering world that is overcoming these problems, namely Python.

“Python is very scalable. Each piece of code can be used countless times as it just needs an input and it will create an output. In this way you could solve a large problem by connecting smaller pieces of code; it is like having digital building blocks”. - Johan Tuls, BAM

Excel, Matchcad, and Python

Peter Konijnenbelt: “Until now we at Arcadis made Excel and Mathcad sheets available with the reason to ensure that all engineers had the same working method to do calculations. But with the introduction of Python we went up to a whole new level: now we calculate a cloud of possible outcomes to find the best one. It also allows us to make a user interface for the tools, where results are presented visually for everyone to use.” This way of working gives more opportunities to find optimal solutions.

Tjeerd Hermsen likes Python because you can command other software and automate tasks, he explains ”we extract cone penetration test (CPT) data stored in Excels sheets and automatically make D-Foundations (geotechnical software) model from it”

Digital Building Blocks

I observed that the common approach to building scalable software and cooperating with other specialists is to think in digital building blocks that can be interconnected. This make the process versatile and scalable. Some of the benefits of using the building block approach are:

• You can quickly build code by utilizing someone else’s work.
• It enables people to share knowledge in the form of code.
• It makes it easier to replace parts of your code.
• It makes growing the code manageable, as each block has a specific function.

“I like it when you can use the work of others. With fewer people you can make more applications if you have a collection of digital building blocks that you can connect” says Johan Tuls. That is also a good reason why people are using Python, explains Van Giessen: “Python is based on digital building blocks, and there are many available for free, such as libraries for numerical calculus or data science”.

Creating new building blocks

Companies are also investing in making their own blocks. “Large software products try to provide a total solution; however, they are not always flexible enough to work efficiently. That is why we are transferring the expertise we have into our own software tools, that add to the existing software and which we can make available to further valorise knowledge”, tells Kelvin Roovers.

“Parametric design will cover a much larger part of the work. In the future you will have modular systems with an application for each part of the construction. The engineer will create, connect, and validate these apps instead of doing the calculations” - Nynke ter Heide, Vidabo

According to Johan Tuls, software development will have a big impact on construction companies. “Construction companies grow linearly, but programming will make it exponential. Companies will accumulate digital building blocks over time, which will be an important asset of the company. With these blocks you can make better applications faster and faster. This will lead to a new business model that has yet to be developed”.

How to make your code accessible for everyone

I have observed that many talented people create models and analysis tools that have a big impact in their organization. As the adoption of their tools grow in the organisation, they fall victim to their own success. These tools are often too complex to be used by other people, and the creator does not have enough time to process all the requests from the organisation, which therefore becomes the bottleneck. Worse, what happens with that knowledge if they leave the organisation?

Controlled distribution

What you want is to distribute the code in a controlled manner. This, however, can be quite challenging when choosing the wrong tools. “Excel does not have a version control. You can download the sheets from the company’s server, but how do you make sure everyone is using the latest version and that upgraded sheets are compatible for each application?”, says Nynke ter Heide. Kelvin Roovers shares a similar experience: “you don’t want to share code in Excel sheets. Everyone starts making their own modifications and it is impossible to keep track of the changes. It is better to keep you code externally, for example using Python”. “There are good solutions for code distribution among programmers when using Python, like GitLab”, says Kevin van Giessen.

User interface required

The reality is that most people who can benefit from using these codes lack the programming skills necessary to understand them. Part of the solution is “a good user interface with visualisations of results. This makes the application accessible to everyone”, says Johan Tuls. It is useful to have a web-based application because version control is guaranteed, explains Kelvin Roovers. “When improving an application, all you have to do is put it online to make sure everyone works with the latest version right away.”

Web-based applications also have tremendous benefits for the end-users. “This allows different users to access a central point from anywhere, at any time, so they work with the same design or version of the information and make decisions together”, explains Kevin van Giessen. He adds, “It makes distribution of your code also very controlled; the administrator can give users permission to access different parts of it and remove it if desired”.

Personally, I think technical people can be very critical about their work and think that it is not ready to be used by others; that there is still too much to improve. This is probably true. However, the solution is not to improve the code by yourself, but to make it available to others with a good user interface. You want people to use your software and give you feedback. This is the best way to truly understand what needs to be improved so that people can use it. It could be anything from technical features to greater managerial awareness. The shorter the feedback loop, the more organically your software grows in the organization, and the more likely people will use it.

Why are engineers using VIKTOR?

Toolbox

Kevin van Giessen explains: “VIKTOR is an application development platform to build professional web-based applications with Python. It’s easy to use; it’s like having a digital building block toolbox to create your own engineering applications. You can integrate with other engineering software, make user interfaces, create 3D models, use map views, perform optimisations and visualize results. This gives engineers the opportunity to become a professional software developer: the engineer developer. Kevin continues, “By enabling the engineer to build user-friendly software applications, knowledge is turned into an asset that can be distributed within and outside the organisation.”

“Distribution is one of the biggest advantages of VIKTOR applications: you put it online and everyone can use it.” - Kelvin Roovers, BESIX

Sharable application

Nynke ter Heide adds “Having Python code is different from an application that you can share with your colleagues”. She continues: “Making a Python script is nice for a proof of concept, but with the VIKTOR platform your code shifts to another level. It’s a way to create applications that everyone can use.”

Unity

Tjeerd Hermsen: “Colleagues started sharing Python scripts while not all using the same version of the required packages, which made the scripts unusable. A lot of time is wasted trying to find and fix such problems. “The solution was to create a VIKTOR application. He explains:” The VIKTOR application ensures that all users are working with the same version of the external Python package. In addition, the Excel files that are used as input for the calculation are used, centralised in the database and they do not have to go back and forth between colleagues.

“It’s amazing that we are able to develop an application within 1 day, which ensures that everybody uses the same analyses and models. This saves time and reduces the chance of errors.” - Tjeerd Hermsen, Sweco

User-friendly

Kelvin Roovers comments: “What I like is that you can program complex applications with a user-friendly interface and easily present results through different visualisations”. “VIKTOR is also interesting for simple applications. There are many Excel files in a company that can be easily converted into a professional application with version control, user management system and visualizations. It is more user-friendly than a multi-sheet Excel file, and you can perform different calculations to find the best solution, ”says Nynke ter Heide. “As a freelancer, I like to be at the beginning of developments. I regularly see that my customers are enthusiastic about my proof of concepts, but that it is challenging to put them into production in the organization. VIKTOR can help meet this challenge and turn concepts into applications, ” she adds.

Web-based and automated

Johan Tuls notes that he wanted to “create and host a web-based application; In the perception of management, however, this differs from the main field of work of the engineer and the company. A partnership with VIKTOR has reduced or overcome this risk ”. Peter Konijnenbelt says: “The management and engineers are very positive about VIKTOR and the results we have achieved in a short period of time”. He adds: “Their team has helped us a lot, you notice that they are much more involved with automation and have a different mindset. They help set up a better parametric model and visualize the results so that we can compare different solutions."

VIKTOR application for designing subways under roads and railways (BAM)

Want to know more?

This is the first of a series of articles about engineering and Python. The next articles will be published soon. We will talk about how you can make parametric design tools using Python and common engineering software like Grasshopper, Dynamo, SCIA, RFEM and IDEA. To write these articles we interviewed engineers that make these kinds of tools on a daily base. Real pros!

Maybe you wonder why we are writing this. At VIKTOR we are on a mission to unleash the world’s real engineering potential. We are developing a platform where engineers can build their own web-based applications using Python. We work closely together with engineering, construction and manufacturing companies and help them boost their digital transformation. We would love to hear your feedback on our ideas, try out the platform, and help us create the perfect tools for you.