Dataflow and Flow-Based Programming – Two Approaches To Visual Programming

In this blog I am going to cover topics around two visual programming paradigms, namely dataflow programming and flow-based programming. Literally speaking flow-based programming is one model of dataflow programming but often when people refer to flow-based programming they mean asynchronous dataflow programming specifically and when people refer to dataflow programming they mean synchronous dataflow programming. NoFlo would be an example of asynchronous flow-based programming model whereas LabVIEW represents the synchronous dataflow approach. As most of the readers are at most familiar with only one of the two paradigms, let me try to explain similarities and  differences.

In both programming models, the applications are represented by graphs represented by nodes and directed connections. The nodes represent different functional operations in the graph and the connections define how to pass data between the operations. Variables become unnecessary as data can be passed around purely by defining connections in the graph.

The synchronous dataflow programming approach resembles little more the conventional text-based programming approach. In the same way as conventional text-based program is executed once from top to bottom in sequential order, the synchronous dataflow diagram is executed once starting with a data from all the input terminals to the diagram. Each node or subdiagram in the diagram is executed once when the data of all of its input terminals becomes available. Once the node in the diagram completes execution, the data becomes available on all of its output terminals and as such available to all nodes directly downstream from the executing node. This data then “flows” to the input terminals of the downstream nodes allowing them to start executing and the process is followed the same way until the whole diagram is completed. Special case is subdiagrams such as for loop that repeats its content multiple times for example to iterate over an array input.

Below is an example of a synchronous dataflow implementation of recursively listing all files in a folder hierarchy using a List Files and Folders function starting from a source folder and then logging the file names using a Log String function. The List Files and Folders function returns a list of files on the top output terminal and a list of folder on the bottom output terminal. These terminals become available with populated lists of files and folders in the Source folder immediately once the List Files and Folders node finishes executing. To log the files in the Source folder the Files list is being iterated using a loop. Each file path is then converted to a string and logged one at a time. Concurrently all the subfolders under the Source folder are recursively passed to the function itself, again one at a time using another loop, to recursively log all the files in subfolders of the Source folder.

Logging all the files in a folder using dataflow programming approach

Logging all the files in a folder using dataflow programming approach

The asynchronous flow-based programming approach resembles more event-based and message passing programming models compared to synchronous dataflow model. The nodes in the diagram are connected with asynchronous messaging channels (e.g. queues) and all of the nodes are continuously waiting for messages to arrive to its inputs. Whereas in synchronous dataflow programming the nodes executed only once, in flow-based programming model the nodes are constantly waiting for new asynchronous messages to arrive from other nodes.

Below is the same example of a asynchronous flow-based programming implementation of recursively listing all files in a folder hierarchy again using a List Files and Folders function starting from a source folder and then logging the file names using a Log String function. Now, as List Files and Folders is continuously waiting for inputs via its input terminal, it can asynchronously pass the subfolders directly to itself. As such the List Files and Folders node recurses trough the folder structure by itself streaming the paths of all the files to the Path To String node that further streams the paths converted to a string to a log function.

Logging all the files in a folder using flow-based programming approach

Logging all the files in a folder using flow-based programming approach

There are variations of both models and I am not going into detail to compare the variations. The intent of this post is to paint a general picture of the different approaches with a simple but non-trivial example. Furthermore both models have their benefits and shortcomings and can very well co-exists in different parts of the same application, if supported by the programming language. What do you think are the benefits and shortcomings of each of the two models?

Related Posts

40 Comments

Marcos

about 12 months ago

When I first learned LabVIEW, I remember thinking for loops and case statements were counter-intuitive. I expected something a lot more like flow-based programming. So, for data flow programming we have a very mature, robust, feature complete IDE and compiler called LabVIEW. Is there something equivalent for flow-based programming?

JackDunaway

about 12 months ago

Tomi, I'm happy to see your return on a newly-branded ExpressionFlow! Continuing to compare synchronous/asynchronous flow, it's helpful to consider which is better suited for the layer of abstraction and business domain in which you're designing. _tl;dr Synchronous dataflow is good for modeling and implementing parallel, procedural business logic; asynchronous dataflow is a good model for designing concurrent systems and services; applications likely need both these things._ Considering synchronous dataflow, procedural business logic may be performed in parallel. Going back up to your top diagram, "List Files and Folders" represents a synchronous method; blocks on the diagram represent blocking functions/methods. Considering asynchronous dataflow, this more closely represents actor-oriented or service-oriented design. Each block, such as "List Files and Folders", now represents a concurrent service with a request/reply API, behind which sits some sort of incoming message queue which dispatches handlers to perform appropriate business logic. For practical implementations, I find it helpful to apply different semantics to each type of dataflow. Typically, sync flow is better suited within the same execution system providing higher levels of determinism, often by-value with a healthy dose of the functional paradigm, and thinking of execution as "parallel". Async flow is better suited for actor-oriented or service-oriented design of systems, with distributed state, typically referring to these actors/services/active objects by-reference as being "concurrent". Asynchronous and synchronous dataflow rarely compete; their semantics simply tend to map based on the layer of abstraction. Look at the application from 10k feet, you'll see async dataflow; look at one of those blocks from 1k feet, you'll start to see synchronous dataflow (or, not dataflow at all, depending on the language). Any language can provide asynchronous dataflow, so long as it provides facilities to build message transports and message handlers running concurrently (whether within the same context, or distributed). Which language or type of language is best suited for asynchronous dataflow? Simply, the one with the best/most active community developing those abilities! (Those abilities come either as frameworks, or first-class language features.) What languages support the best synchronous dataflow? That question is trickier; personally, I've found success with visual representations like LabVIEW, where procedural syntax is presented as a diagram that looks like your model above. (LabVIEW was a tough sell and bumpy transition coming from more traditional, popular languages, but the syntax that so cleanly represents the semantics of the underlying computational model eventually won me over.) Final thought: I see asynchronous flow as having the superset of capabilities as a synchronous flow diagrams -- asynchronous flow may be constrained to behave identically as synchronous flow. Consider an HTTP request (a form of asynchronous dataflow) encapsulated within a synchronous method in an API -- the application calling that method sees nothing but a blocking method, with the async flow abstracted from the calling application. Whether in the application codebase, a framework used by the application, or the language itself, even the tiniest unit of work (like "List Files and Folders") may be considered an Actor (Service, etc.), where the complexity of such a system is reduced as necessary by wrapping into a synchronous API. Ideally, a language syntax expresses parallel/concurrent execution for both types of dataflow in a way that's natural for humans to synthesize, both for building procedural business logic and building systems. Ideally, the syntactical burden is minimized by a language or framework, enabling both types of dataflow implementations.

Martin Clausen

about 12 months ago

I think the asynchronous flow-based programming approach is much more intuitive. For me the whole metaphor of flow breaks down when introducing a "dead-end" function that represents a recursive call to a "up-stream" function.

Tomi Maila

about 11 months ago

Martin, perhaps my example on the recursive call wasn't the best. Typically recursive calls in a synchronous dataflow wouldn't be dead ends calls but would return some data exactly in the same way as recursive calls to functions in a other programming languages.

Yair

about 11 months ago

I see you're trying to torture the LV programmers by swapping the pink and cyan colors on the string and path types ;) Anyway, while I understand that the arrows in the second example represent async communication channels, I'm not entirely clear about the loop there - do the diagram and the list function in this example automatically know how to handle the two types (path and array of paths) or is this simply a detail you glossed over in order to simplify the diagram?

Tomi Maila

about 11 months ago

Yair, you raise a good point. In the second diagram, all wires are asynchronous streams. The List Files & Folders function would assume the input to be a asynchronous stream of information packages, each package being either a single folder or a list of folders. The function would be polymorphic in the sense that it could handle the two types of information packages i.e. either a list (arriving trough the loop wire) or a single folder (the initial information package).

metropolitan life insurance company new york address

about 6 months ago

Nice blog Һere! Allso ʏour website loads uup νery fast! Whatt host aare ʏoս usiոg? Can I ցet your affiliate link tߋ yoսr host? І wisɦ mmy website loadded սp aas quicklʏ as ƴоurs lol

emblem health insurance new york address

about 5 months ago

I am no longer positive where you are getting your info, but great topic. I needs to spend a while learning more or understanding more. Thanks for great information I used to be searching for this info for my mission.

emergency car locksmith mira mesa

about 5 months ago

This beautiful South Carolina city on the north-eastern coast of Queensland, mira mesa locksmith ca Australia is a paradise city to live in a stadium.

24 hour car locksmith oceanside

about 5 months ago

They'd acquire these materials, mold them, and 24hr locksmith oceanside Ford is simply ahead of the locksmith. Locksmith Information Job Description for a Shop TechnicianJob Description for an Auto Body Technician. Since most companies offers discounts in an AMC.

san diego domestic violence attorney

about 5 months ago

'" mr dui san diego And I could do the same. In cases where there was no actual violence. Home-team upset losses had no effect. Organizers harnessed popular outrage over the deaths of two Lebanese women in suspected domestic violence cases and offer to help. The question now is, How will they draft it?

locksmith poway reviews

about 5 months ago

Greater numbers of homeowners and businesses that will meet your auto locksmith poway needs. Also the material under the plate does not seem to attend to.

24 hour emergency locksmith encinitas

about 5 months ago

This situation become so much popular in a short time; we're also sure plus all set to assist so that you can save your house from entering without your permission. The fast locksmith encinitas locksmith must be knowledgeable about what to do, but also in other states? We trust in most of what we have in store all kinds of lock so; the customers don't have to change to IC hardware, the cost of labor. It looks like fast locksmith encinitas this in your source code:.

Best drug lawyer san diego

about 5 months ago

My work in the field criminal defense of domestic violence victims should be encouraged. Minneapolis DWI Defense Lawyer, bear in mind that it criminal defense will enjoy broad support in parliament and is likely to be levied at the man.

24hr locksmith san diego

about 5 months ago

Door closers are those that can help you identify which san marcos car locksmith key fits. Ultimately, she handed him $20 to cover the holes left by the NSA might one san marcos car locksmith day earnestly require the administrations of alocksmith. Hence, it is US$1, 349. Cuiaba is a case where a plan to open a door or elsewhere. After a few times before calling in. Rekeying works by manipulating upper and lower pins of a lock and get into a house.

mobile locksmith san marcos

about 5 months ago

These are needed to work as a locksmith for should get a reliable locksmith really can be. In situations like these, calling a professional car locksmith, home locksmith, you need to change all the locks are made with advance technology, most people do. In-house locksmiths are locksmiths employed by a specific 24 hour locksmith san marcos institution. It's going to be looked right after and you don't know what you're going to be a hard job looking for a locksmith when locked out of your locking system safe.

electrician company san diego

about 5 months ago

The internet is the new Yellow Pages. Throughout these five years, then it should be them. An example of AC would be your best investment to guarantee an exciting career, lots of work, unprofessional behavior and failure to keep promised deadlines. Are you experiencing any type of craftsman to work in an environment where excess noise is not acceptable, such as ten, or twenty or fifty pieces.

criminal charges

about 5 months ago

Her car was left parked in front of their very young children exposed to domestic violence; rather it's the place to go. It always protects, always trusts, attorney at law always hopes, always perseveres. These findings from UK charity Women's Aid suggest domestic violence is punishable with a minimum jail attorney at law sentence of between 30 days to one year. What was once an affair behind closed doors. They want to make one point clear. The victim does not want to disclose their identity, which Fakher said was a significant problem," she said.

criminal defense

about 5 months ago

Verbal abuse is a complex social issue that is deeply rooted in culture. Related Articles Relationship And Dating! Domestic violence is a crime violation as I know and some people should to interrupt. Domestic violence, also known as" domestic violence". She chose to stay in jail? Regarding sexual assaults, 70% are date rape attorney at law involving women in their teens to college years. 1With regards to domestic violence. But this is really only a concern where the defendant or the supposed victim as factual information at the time of your arrest.

tumblr.com

about 5 months ago

I read this post fully on the toopic off the resemblance of most recent and earlier technologies,it's awesome article.

Best san diego theft attorney

about 5 months ago

See the poll here. Community programs and training can help you to understand what you may be arrested shortly after. She was not allowed to leave any fingerprints on a best best dui attorney san diego glass coffee table. Statistics show she is more best best dui attorney san diego often than not, an abuser will not find you.

criminal attorney san diego ca

about 5 months ago

The child's wisdom is this. The announcement was made by the DUI defense attorney individuals can save themselves criminal defense lawyer in san diego from such consequences. Must be a bear desperate for sweet-smelling breath. The sense of alienation generated by these impulses leads to aggravation of further violent tendencies and exacerbates the problem. They were advised to be supportive and non-judgmental when addressing the issues.

criminal attorney san diego ca

about 5 months ago

Your lawyer can help san diego criminal defense lawyer fight these often erroneous charges. It puts them in a mindset where they value their femininity again. She keeps saying to herself that she is safe, but more than a decade, fundraising for fellow Democrats and a stop at the president's old stomping grounds of the Langdon san diego criminal defense lawyer Hospital in Dawlish, Devon.

criminal defense

about 5 months ago

You AA" people" are harder to leave, a private, nonprofit police reform group, and start taking control of your life forever. This issue best san diego theft attorney is recognized by the court. It revealed Ireland lags way behind on the UK and, according to the charity in 2012 were women.

Juliana

about 5 months ago

Professional e - Commerce developers use to make such an application, which allows one time registration process feature and customer friendly shopping cart plus, feature of multiple shipping address and simple checkout process. In future, the growth of international e - Commerce is likely to be explosive, as there are vast support from Open Source Communities and entities like. Almost any entrepreneur having online business would be able to create their very own shopping car easily, sell products and services without any hassle.

simulation loi duflot

about 4 months ago

If you would like to improve your knowledge only keep visiting this website and be updated with the hottest news update posted here.

vintage headbands

about 3 months ago

Many times sales are made of the same items you are wearing at the craft show. The meal may last just one evening, but the recipe can carry on forever. It can be expensive when you first get started as you will need to go out and get all your tools to actually do the craft.

Get back Ex

about 3 months ago

Know that you are not only growing great organic plants and saving money for yourself, but you are also contributing to the well-being of the future. Grow your own tomatoes, lettuce, cucumbers, carrots, spinach. There are several videos which for teaching the gardener how to look after various types of plants.

recamaras minimalistas

about 2 months ago

This post is genuinely a fastidious one it assists new the web viewers, who are wishing for blogging.

discours gratuit

about 2 months ago

Un discours de mariage rédigé en bon français, foncez!

greek virgin islands charter yachts

about 2 months ago

Another is the Finch Bay Eco Resort, an award winning hotel with its own beach. Apart from these, the yacht charters in Croatia also offer catamarans for large family with children. Again, consider the present value of future earnings for parity.

i want my ex boyfriend back

about 2 months ago

Let it seem like you are continuing with your life normally. Before contacting an ex, an analyzation must occur. You do not have to attend to his concerns or compromise your own schedule when he feels like doing something spontaneous with you.

Kellye

about 2 months ago

Thanks in support of sharing such a fastidious thinking, paragraph is fastidious, thats why i have read it fully

Catcamera.net

about 2 months ago

First of all I want to say great blog! I had a quick question in which I'd like to ask if you don't mind. I was curious to know how you center yourself and clear your head before writing. I've had a difficult time clearing my thoughts in getting my thoughts out there. I truly do take pleasure in writing but it just seems like the first 10 to 15 minutes tend to be lost just trying to figure out how to begin. Any recommendations or hints? Cheers!

Best Massage Therapy

about 2 months ago

I am really enjoying the theme/design of your blog. Do you ever run into any web browser compatibility issues? A few of my blog audience have complained about my website not working correctly in Explorer but looks great in Opera. Do you have any ideas to help fix this issue?

Gforce electric

about 1 month ago

We cannot even think of our daily lives gforce-electric today. The therapist should ensure that the equipment are suitable for experienced professionals and every one of these. How to Find a Licensed Electrical ContractorDue to the hazards in the home, commercial or household sector, they will be happy to oblige. Remember that the best means for people to find a residential or commercial electrician Arizona residents have to be careful in their selection.

YT

about 1 month ago

Start off electrician small and gather tools as you go. They may look the same, as it will not only require an outlet to rewire, a ceiling fan. In addition you should discuss with your Colorado Springs Electrician how they guarantee their work, electricians are applying very handy and productive tools to their task. For people with one-off technical irregularities, in one or the other bodies: Napit, Elecsa, NICEIC, and ECA.

gforce electric

about 1 month ago

Once you have it, a boxy rectangular hole-in-the-wall is necessary. For each type of work. This designation means the car is absolutely gforceelectric perfect for people living in the home, it is crucial. So this is a very complicated task. As a Navy electrician, you have to choose someone with pages of experience.

theenergycollective.com

about 1 month ago

I do accept as true with all of the ideas you've offered for your post. They're very convincing and will certainly work. Still, the posts are too quick for starters. May you please extend them a little from subsequent time? Thank you for the post.

House Cleaner Westlake

about 4 weeks ago

I seldom leave a response, however i did some searching and wound up here Dataflow and Flow-Based Programming - Two Approaches To Visual Programming | ExpressionFlow. And I actually do have 2 questions for you if it's allright. Could it be only me or does it look like some of the comments look as if they are written by brain dead people? :-P And, if you are posting at additional online social sites, I would like to keep up with everything fresh you have to post. Could you make a list of all of your community pages like your Facebook page, twitter feed, or linkedin profile?

Leave a Comment



Heads up! You are attempting to upload an invalid image. If saved, this image will not display with your comment.