Tutorial
DSP
KUIML
How-to
Scripts
How-toShare data between instances
February 13, 2023

How to share data between instances

If you want your plugins to communicate you can achieve it in various ways. We'll show two possible solutions tested in our projects:

  1. Using built-in "model" mechanism (GUI-side);
  2. Using shared memory (DSP side, C/C++);

Using built-in "model" mechanism

Every Blue Cat Audio plugin has a feature that allows sharing data between instances of the same plugin (on the GUI side). It's not documented and is sometimes tricky to use in Plug'n Script, because the "model" works on the plugin level and in Plug'n Script we work with different scripts doing different jobs, so it's like having many different plugins. So it's a little tricky. But with some effort you can make it work and it can do some amazing things.

Maybe you've noticed that in the Plug'n Script skin you can select the default editor for editing script, kuiml or opening the log files. And when you change it, it applies to all instances of Plug'n Script. If you take a look inside the skin, the name of the editor is kept in the string, for example: "global.edit_script_application". If you try displaying it and changing it manually, you'll see that it updates immediately in all instances of the plugin.

  1. <TEXT value="global.edit_script_application:" h_align="left" opacity="0.5" />
  2. <TEXT_EDIT_BOX string_id="global.edit_script_application" width="300" change_on_type="true" />

So this is definitely something we can use to communicate between instances.

Model folder location

The file where this string is declared is called "global.model" and is located inside the Plug'n Script plugin in the "Model" folder.

The path on Mac for PnS VST3 version is (use 'Show Package Contents'):
/Library/Audio/Plug-Ins/VST3/BC Plug'n Script VST3.vst3/Contents/Resources/BC Plug'n Script VST3 data/Model/

On Windows the location is similar:
C:\Program Files\Common Files\VST3\BC Plug'n Script VST3 data\Model\

* For each version of the Plug'n Script (VST2, VST3, AU, AAX, Standalone) there's a dedicated "Model" folder, so find the right one for the version you're using.

Let's add a new param into the global.model file:

  1. <?xml version="1.0" encoding="utf-8" ?>
  2. <MODEL id="global">
  3. <!-- these strings are declared by Blue Cat Audio -->
  4. <INCLUDE_ONCE file="global.$_SYSTEM_TYPE_$.inc"/>
  5. <STRING id="edit_script_application" default="$DEFAULT_SCRIPT_EDITOR$" exposed="true" persistent="true"/>
  6. <STRING id="edit_kuiml_application" default="$DEFAULT_SCRIPT_EDITOR$" exposed="true" persistent="true"/>
  7. <STRING id="open_log_application" default="$DEFAULT_SCRIPT_EDITOR$" exposed="true" persistent="true"/>

  9. <!-- we add another global param -->
  10. <PARAM id="common_gain" min="-20" max="20" default="0" unit="dB" exposed="true" persistent="true" />
  11. </MODEL>

After we've updated the model we have to reload the plugin (sometimes it's easier to reload the whole project or restart the host).

After reloading the plugin we can access this parameter in our skin like this:

  1. <PARAM_TEXT_CONTROL param_id="global.common_gain" />

Note that this will be available in the main skin (the .xml file located in the Skins folder). But if you don't use it in the main skin it will be "stripped" and will not be visible in the "subskin" (.kuiml file). To prevent stripping this parameter we can add this line to the main skin:

  1. <REQUIRED_OBJECTS object_ids="global.common_gain" />

Or we could also use a mask like this (but it can be slow if we have a lot of objects):

  1. <REQUIRED_OBJECTS object_ids="global.*" />

Now we can use this shared global parameter in our plugin in .kuiml as well.

  1. <TEXT value="Shared gain" />
  2. <PNS_BLACK_VINTAGE_B_KNOB param_id="global.common_gain" />
  3. <PARAM_TEXT_CONTROL param_id="global.common_gain" />

Note that we've declared this parameter with persistent="true" - this makes it restore its state even when the project is reloaded. Using exposed="true" is required for it to be seen in the main skin and not only in the model itself.

Three models

If you examine other Blue Cat Audio plugins you'll see that there can be up to three different models in Model folder.

  • shared.model - parameters and strings declared inside are shared between instances, so it's like a communication layer
  • global.model - almost like shared.model, but it also can keep values persistent between sessions - useful as a place for global preferences
  • model.model - parameters and strings are not shared, so here you can keep things that can be different in each instance

To get some ideas on how you can use these models, take a look at Blue Cat's Gain Suite. Install the plugin and examine its "Model" folder. You'll see how parameters can be linked, unlinked, so that plugin instances are grouped. Also many other BCA plugins use this "Model" thing, so check them out to learn.

Advanced possibilities

This built-in "model" allows you to also run actions via ACTION_TRIGGER, use TIMER, execute scripts and do a lot of other stuff that you usually can't do when the GUI is closed. So even if GUI is closed and DSP is bypassed, you can still do some work in the Model. Though it's not recommended to do heavy processing here, because the model thread is shared between all instances of a plugin.

Exporting

The default skin for Plug'n Script doesn't support exporting model files (when building a plugin). So you can either manually copy the Model folder into your exported plugin, or use LM Skin. When you change your model files it detects that and shows export model options in the Export window.

You can also use a "trick" to keep model files separated for each particular script and for proper exporting. Say your main script is called "comp.cxx". You can create "comp.model", "comp.model.global" and "comp.model.shared" in the same folder where the script is. Then in the PnS "Model" folder create additional "global-custom.model", "model-custom.model" and "shared-custom.model" files.

In the global.model file keep all the code that is related to all scripts, and in the end of it add an include:

  1. <INCLUDE_ONCE ignore_missing="true" file="global-custom.model" />

And in "global-custom.model" include the actual file in the script's folder:

  1. <INCLUDE_ONCE ignore_missing="true" file="/PNS/Scripts/Comp/comp.model.global" />

The same you can do for model.model and for shared.model.

After the export LM Skin renames the comp.model.global from the script's folder to global-custom.model and puts it into the Model folder of the exported plugin (together with global.model - remember that global.model includes this global-custom.model). The same happens for the shared.model and for model.model. So in the exported plugin you have the model files required for this particular plugin, and your model files are easy to access and edit.

Downsides of using a model

The main problems are that it requires reloading the plugin after update and that it is very tricky to debug. If there's an error in the model, it just will not load (without any error messages) and then you usually get a lot of errors in the main skin that can't find some model objects. For getting some messages from the model you can create a string in the model and display it in the main skin, so that you can have some feedback while developing/debugging. Also for linking model params to DSP params you have to deeply understand parameters mapping and how they differ inside PnS and when exported.

Also the communication is not instant. It usually takes from 10 milliseconds for parameters to update. Also remember that the parameters are shared only between instances of the same plugin (and within the same DAW/project/plugin format).

In general a "model" is a very powerful gem in Plug'n Script and other Blue Cat Audio plugins that can help you in various ways, mostly related to the GUI-side of the plugin.

Using shared memory (DSP side)

If you've already switched to C/C++ (from AngelScript), you can try various ways to access memory. Scripts written in AngelScript don't have a direct access to memory, so the only way to commucate with the "outer world" seems to be reading and writing to files. (We also did some experiments opening virtual COM-ports as files, but we wouldn't recommend it as a reliable option).

In C/C++ you can making a variable "static" hoping that it will be shared between instances. But each time PnS plugin loads your compiled code it's renamed and opened as a new library, so "static" will not work as expected.

One of the options that Blue Cat Audio recommends is making another dynamic library that you can load, and use it as a place for sharing data. We tried that and it works, but there's another way we've found that doesn't require building additional libraries - opening virtual files in shared memory.  

Opening a virtual file in shared memory

In this demo we've created a virtual file in shared memory (allocated some space there), got an address of the first byte of that shared memory and used it to share a simple value. See more comments and explanations in the code.

Here we share the routines that can be used to access that shared memory on both Mac and Windows.

Contents of the main dsp file (test_shared_mem.cpp)

  1. // include required stuff
  2. #include <cmath>
  3. #include <string>
  4. #include <vector>
  5.  
  6. // include factory Blue Cat Audio libraries
  7. #include "../library/dspapi.h"
  8. #include "../library/cpphelpers.h"
  9.  
  10. // include LetiMix library to add params in a cleaner way
  11. // see https://pns.letimix.com/how-to/add-params-more-conveniently
  12. #include "../library/params.h"
  13.  
  14. // include shared memory functions (see below)
  15. #include "shared_memory.h"
  16.  
  17. // script name
  18. DSP_EXPORT string name="Shared memory";
  19.  
  20. // placeholders for input and output params indexes
  21. int GAIN, SHARED_GAIN;
  22.  
  23. // to prevent adding params twice
  24. bool params_were_initialized = false;
  25.  
  26. // initialize (usually done once on plugin loading)
  27. DSP_EXPORT bool initialize(){
  28. if (!params_were_initialized) {
  29. // add input params
  30. GAIN = ip("Local", "dB", -20, 20, 0, ".2");
  31.  
  32. // add output params
  33. SHARED_GAIN = op("Shared", "dB", -20, 20, 0, ".2");
  34. }
  35.  
  36. // try to initialize shared memory (see shared_memory.h)
  37. if (initSharedMemory() == false) {
  38. print("Shared memory unavailable!");
  39. return false;
  40. }
  41.  
  42. return true;
  43. }
  44.  
  45. ///////////////////////////////////////////////
  46. // now functions executed when the plugin is running
  47.  
  48. // read input param changes
  49. DSP_EXPORT void updateInputParametersForBlock(const TransportInfo* transportInfo) {
  50. double gain = IP[GAIN];
  51. if (sh != nullptr) {
  52. // set value in shared memory
  53. sh->gain = gain;
  54. // print("updated to " + std::to_string(sh->gain));
  55. }
  56. }
  57.  
  58. // output values
  59. DSP_EXPORT void computeOutputData() {
  60. if (sh != nullptr) {
  61. // output value from shared memory
  62. OP[SHARED_GAIN] = sh->gain;
  63. }
  64. }
  65.  
  66. ///////////////////////////////////////////////
  67. // when the plugin is unloading
  68.  
  69. // shutdown shared memory (see shared_memory.h)
  70. DSP_EXPORT void shutdown() {
  71. shutDownSharedMemory();
  72. }

Contents of the shared_memory.h

  1. // THIS IS A DEMO LIBRARY FOR USING SHARED MEMORY
  2. // Jan 2023 // Ilya Orlov // support@letimix.com
  3.  
  4. // Down below be add a function called 'sharedMem_getAddr'
  5. // implemented for both Mac and Windows machines.
  6. // This function gets a 'virtual file name' and
  7. // size in bytes (the amount of shared memory you need).
  8. // On success it returns a pointer to start of that shared memory block.
  9.  
  10. // You can use this memory the way you want,
  11. // for example use that start address (memAddr) + offset to read or write
  12. // using something like memset or memmove
  13.  
  14. // But it's convenient to make a "structure" and place it
  15. // in the beginning of that shared memory block
  16. // so you can later read and write easily like this
  17. // sh->gain = 0.5; or double freq = sh->freq;
  18. struct sharedParamsStruct{
  19. double gain = 0;
  20. double freq = 0;
  21. double some_data[512];
  22. };
  23.  
  24. // this will be a pointer to that sharedParamsStruct in shared memory
  25. // so we could use it like sh->gain = 10;
  26. sharedParamsStruct * sh = nullptr;
  27.  
  28. // variables for shared memory allocation
  29. int memSize = 0; // will get the value after allocation
  30. void* memAddr = nullptr; // will get the value after allocation
  31.  
  32. ////////////////////////
  33. // SOME HELPERS WE USE
  34.  
  35. // PnS print implementation with std::string
  36. void print(std::string s){
  37. if (s.length() > 0)
  38. print(s.c_str());
  39. }
  40.  
  41. /////////////////////////////////////////////////////////////
  42. // NOW LET'S IMPLEMENT A FUNCTION TO GET THAT SHARED MEMORY
  43. /////////////////////////
  44. // The implementation is different on Windows and Mac, but the result
  45. // is the same - you get an address in shared memory where
  46. // you can read and write.
  47. // Notice that this address will be different for each plugin instance.
  48. // Because it's not the "real" address, it's translated by the OS.
  49. // But it works like you're accessing the same memory.
  50. ////////////////////////
  51.  
  52. // for Mac
  53. #if defined(__clang__)
  54.  
  55. // MAC
  56. #include <sys/mman.h>
  57. #include <sys/fcntl.h>
  58. #include <errno.h>
  59. #include <unistd.h>
  60.  
  61. int fdMapFile = -1;
  62. void* pMemAddr = nullptr;
  63. size_t _memSize = 0;
  64.  
  65. // returns pointer to shared memory addess
  66. void* sharedMem_getAddr(const char* virtualFileName, int & sizeToAllocate){
  67. // get page size (or granularity, which is bigger) on current system
  68. double pageSize = 0.0 + sysconf(_SC_PAGE_SIZE);
  69.  
  70. // update sizeToAllocate
  71. sizeToAllocate = pageSize*ceil((0.0 + sizeToAllocate)/pageSize);
  72.  
  73. // get shared memory file descriptor
  74. fdMapFile = shm_open(virtualFileName, O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
  75. if (fdMapFile == -1) {
  76. print("shm_open error");
  77. pMemAddr = nullptr;
  78. return pMemAddr;
  79. }
  80.  
  81. // extend shared memory object as by default it's initialized with size 0
  82. ftruncate(fdMapFile, sizeToAllocate);
  83.  
  84. // map shared memory to process address space
  85. pMemAddr = mmap(NULL, sizeToAllocate, PROT_WRITE|PROT_READ, MAP_SHARED, fdMapFile, 0);
  86. if (pMemAddr == MAP_FAILED) {
  87. print("mmap error. sizeToAllocate: " + std::to_string(sizeToAllocate));
  88. pMemAddr = nullptr;
  89. return pMemAddr;
  90. } else {
  91. _memSize = sizeToAllocate;
  92. }
  93.  
  94. return pMemAddr;
  95. }
  96.  
  97. // on closing
  98. void sharedMem_onShutdown(){
  99. if (pMemAddr != nullptr) {
  100. munmap(pMemAddr, _memSize);
  101. }
  102. if (fdMapFile >-1) {
  103. close(fdMapFile);
  104. }
  105.  
  106. // we could also delete the virtual file, but we don't have to
  107. // shm_unlink(virtualFileName);
  108. }
  109.  
  110. #else
  111. // WINDOWS
  112. #define WIN32_LEAN_AND_MEAN 1
  113. #include <windows.h>
  114.  
  115. HANDLE hMapFile = 0;
  116. void* pMemAddr = nullptr;
  117.  
  118. // returns pointer to shared memory address
  119. void* sharedMem_getAddr(const char* baseFileName, int & sizeToAllocate){
  120. // define real size to allocate depending on page size
  121.  
  122. // get page size (or granularity, which is bigger) on current system
  123. double pageSize = 0;
  124. SYSTEM_INFO system_info;
  125. GetSystemInfo(&system_info);
  126. pageSize = system_info.dwPageSize;
  127. if (system_info.dwAllocationGranularity > pageSize) {
  128. pageSize = system_info.dwAllocationGranularity;
  129. }
  130.  
  131. // update sizeToAllocate
  132. sizeToAllocate = pageSize*ceil((0.0 + sizeToAllocate)/pageSize);
  133.  
  134. std::string virtualFileName = std::string("Local\\") + baseFileName;
  135.  
  136. // get handle to virtual file
  137. hMapFile = OpenFileMapping( FILE_MAP_ALL_ACCESS, FALSE, virtualFileName.c_str());
  138. if (hMapFile == nullptr) {
  139. hMapFile = CreateFileMapping(INVALID_HANDLE_VALUE, NULL, PAGE_READWRITE, 0, sizeToAllocate, virtualFileName.c_str());
  140. }
  141. if (hMapFile == nullptr) {
  142. print("Could not create file mapping object. Error: "+std::to_string(GetLastError()));
  143. pMemAddr = nullptr;
  144. return pMemAddr;
  145. }
  146. // map view of virtual file in memory
  147. pMemAddr = (void*) MapViewOfFile(hMapFile, FILE_MAP_ALL_ACCESS, 0, 0, sizeToAllocate);
  148. if (pMemAddr == nullptr) {
  149. print("Could not map view of file. Error: "+std::to_string(GetLastError()));
  150. CloseHandle(hMapFile);
  151. pMemAddr = nullptr;
  152. return pMemAddr;
  153. }
  154.  
  155. return pMemAddr;
  156. }
  157.  
  158. // on closing script
  159. void sharedMem_onShutdown(){
  160. UnmapViewOfFile(pMemAddr);
  161. CloseHandle(hMapFile);
  162. }
  163.  
  164. #endif
  165.  
  166. ////////////////////////
  167.  
  168. // initialize shared memory
  169. bool initSharedMemory() {
  170.  
  171. // prevent double initialization of shared memory
  172. if (sh == nullptr) {
  173.  
  174. // for the 'virtual file name' you may also want to add
  175. // name of the current app (DAW), current userid and other stuff
  176. // this way you'll have different 'shared memories'
  177. // when using plugin in different DAWs on the same computer
  178. // or when using multiple user accounts on the same machine
  179. std::string virtualFileName = std::string("my_virtual_file");
  180.  
  181. // how much memory we need
  182. int extra_memory = 16384; // in addition to sizeof sharedParamsStruct
  183. int memSizeToRequest = sizeof(sharedParamsStruct) + extra_memory;
  184.  
  185. // ask for memory
  186. memSize = memSizeToRequest;
  187. memAddr = sharedMem_getAddr(virtualFileName.c_str(), memSize);
  188. // if we got a memory address, it's a success
  189. if (memAddr != nullptr) {
  190. // place the "sh" pointer at the start of that memory
  191. sh = (sharedParamsStruct *) memAddr;
  192. } else {
  193. sh = nullptr;
  194. }
  195.  
  196. // output some debug info
  197. print("Virtual file name: '" + virtualFileName + "'; requested: " + std::to_string(memSizeToRequest) + "; allocated: "+ std::to_string(memSize) + " bytes");
  198. }
  199.  
  200. if (sh == nullptr)
  201. return false;
  202. else
  203. return true;
  204.  
  205. }
  206.  
  207. // on unloading the plugin
  208. void shutDownSharedMemory(){
  209. // release what we're using
  210. sharedMem_onShutdown();
  211. }

Sending audio using shared memory

One of the things you can do using shared memory is send audio data from one plugin to another. At first it seems like an easy task - just copy samples from the current block in the sender to the shared memory, and in receiver copy these samples from shared memory to the current block samples.

It would actually work in some DAWs, but in other DAWs it will not, because the block size in sender and receiver can be different, and block size can also change during automation and in other situations. So the solution is to have a buffer in shared memory long enough for the largest possible block size (or maybe x2 or x3 of that) and just to copy samples to the end when you have them in the Sender. And in Receiver keep track of the last reading position in the buffer and read as many samples as you need for the current block.

When you get to the end of the buffer, jump to the start: this cyclic writing works much faster than moving the whole buffer down in memory to add another block at the end - don't do it, it's slow and hard on CPU.

If in Receiver you need to add a delay to compensate for the delay between Sender and Receiver, you can either make the shared memory buffer long enough, or add another (larger) buffer in the receiver that can enlarge depending on the required delay.

Also be aware of threading problems, which can occure when sending audio from one channel to another, and the channels are unrelated. DAW can process them on different threads (and on different CPU cores), so that in Sender you can have already say 7 blocks processed, and in Receiver only 1, or vice versa (which is worse). In that case you may also need to add a delay in Receiver so that it always has enough audio to read.

In general sending audio is an interesting and sometimes challenging topic, feel free to share your thoughts and experience in the comments.