vvvv-ZeroMQ
| Author: | velcrome |
| Date: | 02 Dec, 2015 |
| Category: | pack |
| Credits: | velcrome |
Description
What started as a easy drop-in replacement for UDP (Network Server) and TCP (Network Server turned into a full fledged toolkit for the 99% of common asynchronous communication styles.
In its most basic use case, you can use two Dealer (Network.ZeroMQ) socket plugins to then Send (Network.ZeroMQ) and Receive (Network.ZeroMQ) bundled spreads of Raw.
Bind one to an endpoint (pretty much like the old Server), and Connect any count of others to the same address (pretty much like the old Client species). From then on, you will get reliable, and ordered output for every Send you bang (pretty much like you expect, but which the naive TCP and UDP could never deliver, for protocol specification reasons). For most patterns this is even full-duplex.
As for a cooler use case, you can set up a Pub-Sub network just as easily. Bind the Publisher (Network.ZeroMQ), but connect all Subscriber (Network.ZeroMQ)s and listen to the topics of interest. ZeroMQ is smart enough to only wire the stuff subscribers are actually interesed in. With just 3 more nodes (as in xpub, xsub, proxy) you can give your clustered vvvv setup similar characteristics as when using MQTT (or mostly any other software with MQ in its name) without using a third-party broker with suspicious patents.
In brief: this toolkit gives you the means to handle advanced networking science with just a few nodes. Be aware, it does not do any security for you, this is something you'd have to stack on top of it.
- Installation
Use this vpm link to install it with microdees great vpm tool.
You can also create a vvvv-ZeroMQ folder in your packs folder, and unzip a release or prerelease from the GitHub release page.
- Technical details
Most of the technical advance of this pack is directly coming from the NetMQ library, which is an open c# reimplementation of the original c++ libs.
Read the manual!
The vvvv socket nodes provided take full advantage of the simplistic modularity of the original design, but take away alot of functional choices to give you instead an easy yet sufficiently complete node interface: Pick two sockets and transmit binary payloads between them. Each socket has it's own subtleties, which can help you, depending on your requirements and design. And hey, there are only so many patterns you can do ;-)
Supported base protocols are InProc (similar to shared mem, but more like a postal service), TCP (the common default for its reliability across networks even under high load) and PGM (some industry thing for massive pubsub). Sadly IPC is not yet supported.
Each Socket runs in its own thread, independent of the vvvv Mainloop. The similarity of sockets allows easy exchange of patterns during runtime. Have a look into the help patches (one for every socket) and spot the subtile differences, so you can master inter-instance and network communication.
Talking ZeroMQ is supported and understood in more than 40+ languages, including python, java, php, erlang, and c. It allows future-proof IoT for various CPU architectures, and it allows you designing your network according to the means of your hardware, instead of designing around bloated industry-commissioned specifications. Currently it might be a smart idea to rapidly prototype your network configuration with vvvv before moving on to more low-level devices.
vvvv-message has been interfaced as well, because it does away with a lot of hassle of converting to and binning correctly any Raw data, because that is what's going to be weird once you exchange actual data within different vvvv instances across cores or even the net.
This pack needed quite some boiler plating to get there, but I am proud all nodes are up to woei gold standard of bin-sizeing now.
- License
Because this is just a smart shim between NetMQ and the vvvv interface (both LGPL), this pack is LGPL too: any use is free of charge, changing source code is liberal. That said, if you use the actual VVVV, licensing fees do apply for commercial work. Same (plus attribution) if you use additional vvvv-Message nodes to your advantage.
- TL;DR
ZeroMQ it is crazy fast, supposedly sometimes even faster than naive TCP. Code is at github.
Comments
Comments are no longer accepted.Please create a new topic in the vvvv beta forum to discuss this contribution.
this is so great! thanks
Velcrome you are a superhero \o/
btw for anyone interested I found a (quite old) blog post that gives a great executive summary on some of the main patterns: http://nichol.as/zeromq-an-introduction
thanks for this didn't manage to make message interface work though raw data are coming, but message receiver (connected to dealer) doesn't see anything
any hint what to check?
finally found some time to fine tune this. Please test, help patches are provided :)
Hey velcrome, look at this thread. https://vvvv.org/forum/streaming-texture-over-network#comment-217378 Strage behaviour here. b34.2 x64 Win10 x64.
never forget to set the Bind toggle in the sockets, so each endpoint is bound to exactly once!
Hi! How it can be used to send/receive messages via UDP (for example OSC)?
it cannot. zeromq at its core can only utilize tcp, ipc and pgm as transport protocol, to make it act reliable (which udp never is, but which you'll only see when your network is under pressure).
for udp based osc, use the standard vvvv issued UDP (Network Client) and have a look into OSC.
This pack has been successfully updated to beta35.7
How can I use multiple Receive (clients) from the same Send ?
@io make sure the bind pin isn't set, I think.
I've not tried using zeroMQ anywhere but in local host, I just tried to get it working to another pc, and I have issues with binding. Opened help pathes for subscriber and publisher, on localhost all good, when I change the IP to the other PC the node goes red and I get 00:07:34 ERR : vvvv.ZeroMQ: Enable of PublisherSocket failed. NetMQ threw an internal exception with tcp://192.168.178.21:4444 Exception of type 'NetMQ.NetMQException' was thrown.
00:07:34 ERR : Exception occured in TMPluginWrapperNode.Evaluate 00:07:34 ERR : Exception of type 'NetMQ.NetMQException' was thrown.
Changing the port doesnt help. Any ideas?
Something of a bug: If you give the Dealer node a Port of zero(0) when Enabled, you get a hard crash of vvvv, no exception dialog, just the Windoze "stopped working" message on win7 and 8, and nothing on Win 10.
@catweasel Make sure if you have a Dealer attached to a Receive, that the IP address you give it is the local machine doing the receiving (not the remote), and set the Dealer Bind pin to 1. I think the idea here is that the input socket is getting bound to the local PC's network adapter. May not be your problem, but you will get the same error if the receive Dealer has a different IP address than the receiving machine's IP when Bind is set.