StormのTopologyをグラフにして見る（手抜き

python strom

StormのTopologyBuilderの設計はいまいち好きじゃないんですが（idで指定なのと、setBolt・setSpoutが分かれているところ）、それはともかく、できあがったTopologyはグラフにしないとパッと理解できないので、Pythonで適当に書いてみました。

    builder = TopologyBuilder()
    builder.setSpout(1, Spout())
    builder.setBolt(2, Bolt()).allGrouping(0)
    builder.graphviz_format()

digraph topology {
1 [shape=box, label="1@Spout x(default)"];
2 [label="2@Bolt x(default)"];
1 -> 2 [label="all"];
}


    class TestWordSpout(Spout):
        pass
    class ExclamationBolt(Bolt):
        pass
    builder = TopologyBuilder()        
    builder.setSpout(1, TestWordSpout(), 10)        
    builder.setBolt(2, ExclamationBolt(), 3).shuffleGrouping(1)
    builder.setBolt(3, ExclamationBolt(), 2).shuffleGrouping(2)
    builder.graphviz_format()

digraph topology {
1 [shape=box, label="1@TestWordSpout x10"];
2 [label="2@ExclamationBolt x3"];
3 [label="3@ExclamationBolt x2"];
1 -> 2 [label="shuffle"];
2 -> 3 [label="shuffle"];
}


    class RandomSentenceSpout(Spout):
        pass
    class SplitSentence(Bolt):
        pass
    class WordCount(Bolt):
        pass
    builder = TopologyBuilder()
    builder.setSpout(1, RandomSentenceSpout(), 5)        
    builder.setBolt(2, SplitSentence(), 8).shuffleGrouping(1)
    builder.setBolt(3, WordCount(), 12).fieldsGrouping(2, Fields("word"))
    builder.graphviz_format()

digraph topology {
1 [shape=box, label="1@RandomSentenceSpout x5"];
2 [label="2@SplitSentence x8"];
3 [label="3@WordCount x12"];
1 -> 2 [label="shuffle"];
2 -> 3 [label="fields[word]"];
}

だいぶわかりやすいですね。

TopologyBuilderは

TopologyBuilder builder = new TopologyBuilder();
        
builder.setSpout(1, new RandomSentenceSpout(), 5);        
builder.setBolt(2, new SplitSentence(), 8)
        .shuffleGrouping(1);
builder.setBolt(3, new WordCount(), 12)
        .fieldsGrouping(2, new Fields("word"));

より

TopologyBuilder builder = new TopologyBuilder();
builder.set("random_sentence_spout", 
        new Node(new RandomSentenceSpout()).setPallalelismHint(5));
builder.set("split_sentence", 
        new Node(new SplitSentence()).setPallalelismHint(8).shuffleGrouping("random_sentence_spout")) ; 
builder.set("word_count", 
        new Node(new WordCount()).setPallalelismHint(12).fieldGrouping("split_sentence", new Fields("word"))) ;

とかのほうがわかりやすいような気がしますが、僕はもう少し抽象的な定義からTopologyを生成して使おうとしているので、出来上がりさえ検証できれば問題ないですね。

コード

InputDeclarerのstreamIdは使わなそうなので実装していません。