Introduction

The way humans have interacted with computers evolved very quickly from physical rewiring, to punch-cards, to text. But it seems like that upward momentum has stopped as editing text has been the dominant form of programming for quite some time. Text-based languages have been useful and certainly haven’t plateaued yet but maybe taking a step in a different direction isn’t a bad idea. I believe that there is a place for serious non-text based (visual) programming languages and many of the ideas that would come with them.

Brief History of Visual Languages

Many visual languages are designed for new/light programmers. While Scratch may come to mind as a visual programming language, I wouldn’t count it as one. Text is the main form of communication in Scratch, except instead of typing it in, you drag blocks of text around. Something more akin to what I’m taking about would be Unreal Engine’s Blueprints, or a shader graph. These editors express the program in much different form than text, but are fairly domain specific.

One divergence from standard languages was DION Systems by Ryan Fleury and Allen Webster. This project opted to not have the user edit text files directly. This allowed for blocks of text to be edited and easily moved around and reconfigured. While I wouldn’t regard their language as being a visual language, it holds many important ideas about code not quite being text. The project showed that many hurdles programmers face can be taken down and that a level of indirection from what the programmer inputs and what is compiled can be quite powerful.

Competitive Market

Fortunately, a lot of new programming languages are coming out with quite a bit of promise. Many of the ones I follow try to focus on safety, productivity, and general quality of life. Somewhat unfortunately, this means that using a visual programming language to make a programmers life easier or more productive will be competing with many much larger projects.

There is plenty of potential for visual programming languages to make programming more productive, easier, faster, or prettier. I just think that coming at it from that angle is a little risky, especially for someone a little newer, like me. If those problems are trying to be solved already, I think that visual programs could try and solve another problem.

Difference

The main problem I aim to solve is that text representations of code poorly represent the nature in which CPUs operate. Text implies that the program should be executed top-down, in order. If you step through an optimized build in a debugger, however, you’ll notice that the current executing line jumps around quite a lot. This is no mistake and is a consequence that the dependency graph formed by the program is much more complicated than the linear sequence of statements the text advertises.

Even though the one-dimensional nature of text isn’t fully accurate, a general two-dimensional is even less so because programs still follow a general stream of instructions. I think a directed acyclic graph (DAG) would be more fitting as a the program could still flow roughly top-down, but jump around a bit when dependencies need to be filled. The visual nature could show these dependencies and help the programmer understand what the compiler is going to do with their program.

Purpose

This level of concern as for what the CPU is going to really do probably only applies to low-level programmers trying to do some optimizations. When doing optimizations, intrinsics can help and probably get you 80% (these numbers are just estimates) of the way to a fully optimized build. Using assembly allows you to get 100% of the way there, but then many people (including me) don’t know assembly well enough to program in it. My hope is to get programs 90% of the way there by making dependency chains obvious and actionable by better representing them.

This language won’t be for programming whole applications, just the parts you want to be quite optimized. You would probably compile to a DLL or LIB rather than an EXE. While I don’t want the editing experience to be abysmal, brevity of code won’t really be a concern. I would want each ‘vertex’ of the DAG to represent either one or no instructions during codegen. Since the language is all about transparency, I don’t want to hide anything, but that will come at a cost.

Examples

One example of where this language could be useful is a fluid simulation I wrote and optimized a bit using SIMD. My current version is in C# so it is far from perfect but I plan to move it to C++ soon and use some actual intrinsics. Here is one of the most important functions in the simulation, pay most attention to the linearSolveIteration function.

The main thing to note here is that the order that we do the arithmetic can determine the length of the dependency chains. Reasoning about the theoretical speed can be difficult in text form as it is hard to see which instructions can be overlapped, which a graphical representation could better demonstrate. I think a visual language could make optimizing this require less trial and error.

I think the software rasterizer I wrote for Handmade Hero is also a good use case as it is fewer, but larger functions. Here is the inner loop of just blitting a bitmap to another bitmap. I wanted to use the function for rotated and scaled bitmaps but it was too large.

Very rough representation of the function

You can imagine that there are few different dependency chains for each of the color channels. Reorganizing the code to be separated by color rather than by operation could better reflect this, but I think that would make patterns harder to spot. I believe that a visual language could let you achieve both which would be very helpful as you have another axis for organization.

Better Editor

Because of the dedicated editor, I think it’s conceivable that some major quality of life features could eventually be added. Instead of constantly prefixing your variables/functions, those annotations could just be properties that can be edited by inspecting the vertex and clicking a checkbox. This could also lead to increased communication between the programmer and compiler as it is less cumbersome. For example, indicating how often you think an if statement is true so the compiler can better optimize.

Some of the really cool features from DION Systems like “slices” could make their way into the compiler. The editor would automatically be modal as you would be editing text far less often, so regular keystrokes would be commands. I will probably base a decent amount of my editor decisions from the game Shapez 2 as that game has stellar controls in my opinion and solves a lot of the problems I think I will have to solve.

Conclusion

Overall, I’m pretty excited to try and make this language. Hopefully I am not trying to do too much at one time. One downside of making a visual programming language is that you also need to make a whole editor, too. I’m trying to make this language as small as possible while still having the capabilities of C. Maybe one day I will try to make a general graph editor so anyone can make a visual language, but that’s probably too far out.

Sorry if this post seemed too rambly, I’m just kinda excited, I’ll try to make it up to you with a more somber and calculated post down the line. If you have any examples of visual programming languages you like interested in those as I wasn’t able to find that many on my own. If you have any thoughts/suggestions about this project in general I would also love to hear those.

Ideas About a New Low-Level Visual Programming Language