1.. _communicate_with_nodes:
2
3Communication Between Graphs
4============================
5
6
7All graph nodes require a reference to a graph object as one of the
8arguments to their constructor. It is only safe to construct edges
9between nodes that are part of the same graph. An edge expresses the
10topology of your graph to the runtime library. Connecting two nodes in
11different graphs can make it difficult to reason about whole graph
12operations, such as calls to graph::wait_for_all and exception handling.
13To optimize performance, the library may make calls to a node's
14predecessor or successor at times that are unexpected by the user.
15
16
17If two graphs must communicate, do NOT create an edge between them, but
18instead use explicit calls to try_put. This will prevent the runtime
19library from making any assumptions about the relationship of the two
20nodes, and therefore make it easier to reason about events that cross
21the graph boundaries. However, it may still be difficult to reason about
22whole graph operations. For example, consider the graphs below:
23
24
25::
26
27
28       graph g;
29       function_node< int, int > n1( g, 1, [](int i) -> int {
30           cout << "n1\n";
31           spin_for(i);
32           return i;
33       } );
34       function_node< int, int > n2( g, 1, [](int i) -> int {
35           cout << "n2\n";
36           spin_for(i);
37           return i;
38       } );
39       make_edge( n1, n2 );
40
41
42       graph g2;
43       function_node< int, int > m1( g2, 1, [](int i) -> int {
44           cout << "m1\n";
45           spin_for(i);
46           return i;
47       } );
48       function_node< int, int > m2( g2, 1, [&](int i) -> int {
49           cout << "m2\n";
50           spin_for(i);
51           n1.try_put(i);
52           return i;
53       } );
54       make_edge( m1, m2 );
55
56
57       m1.try_put( 1 );
58
59
60       // The following call returns immediately:
61       g.wait_for_all();
62       // The following call returns after m1 & m2
63       g2.wait_for_all();
64
65
66       // we reach here before n1 & n2 are finished
67       // even though wait_for_all was called on both graphs
68
69
70In the example above, m1.try_put(1) sends a message to node m1, which
71runs its body and then sends a message to node m2. Next, node m2 runs
72its body and sends a message to n1 using an explicit try_put. In turn,
73n1 runs its body and sends a message to n2. The runtime library does not
74consider m2 to be a predecessor of n1 since no edge exists.
75
76
77If you want to wait until all of the tasks spawned by these graphs are
78done, you need to call the function wait_for_all on both graphs.
79However, because there is cross-graph communication, the order of the
80calls is important. In the (incorrect) code segment above, the first
81call to g.wait_for_all() returns immediately because there are no tasks
82yet active in g; the only tasks that have been spawned by then belong to
83g2. The call to g2.wait_for_all returns after both m1 and m2 are done,
84since they belong to g2; the call does not however wait for n1 and n2,
85since they belong to g. The end of this code segment is therefore
86reached before n1 and n2 are done.
87
88
89If the calls to wait_for_all are swapped, the code works as expected:
90
91
92::
93
94
95       g2.wait_for_all();
96       g.wait_for_all();
97
98
99       // all tasks are done
100
101
102While it is not too difficult to reason about how these two very small
103graphs interact, the interaction of two larger graphs, perhaps with
104cycles, will be more difficult to understand. Therefore, communication
105between nodes in different graphs should be done with caution.
106
107