{"id":7628,"date":"2015-09-23T04:59:18","date_gmt":"2015-09-23T04:59:18","guid":{"rendered":"https:\/\/unknownerror.org\/index.php\/2015\/09\/23\/bkirwi-coast\/"},"modified":"2022-08-30T15:42:26","modified_gmt":"2022-08-30T15:42:26","slug":"bkirwi-coast","status":"publish","type":"post","link":"https:\/\/unknownerror.org\/index.php\/2015\/09\/23\/bkirwi-coast\/","title":{"rendered":"bkirwi\/coast"},"content":{"rendered":"

In this dark stream-processing landscape, coast<\/code> is a ray of light.<\/p>\n

Why coast<\/code>?<\/h2>\n
    \n
  • \n

    Simple:<\/strong> coast<\/code> provides a simple streaming model with strong ordering and exactly-once semantics. This straightforward behaviour extends across multiple machines, state aggregations, and even between independent jobs, making it easier to reason about how your entire system behaves.<\/p>\n<\/li>\n

  • \n

    Easy:<\/strong> Streams are built up and wired together using a concise, idiomatic Scala API. These dataflow graphs can be as small or as large as you like: no need to cram all your logic in one big job, or to write a bunch of single-stage jobs and track their relationships by hand.<\/p>\n<\/li>\n

  • \n

    Kafkaesque:<\/strong> coast<\/code>‘s core abstractions are patterned after Kafka\u2019s data model, and it\u2019s designed to fit comfortably in the middle of a larger Kafka-based infrastructure. By taking advantage of Kafka\u2019s messaging guarantees, coast<\/code> can implement exactly-once semantics for messages and state without a heavy coordination cost.<\/p>\n<\/li>\n<\/ul>\n

    Quick Introduction<\/h2>\n

    coast<\/code>‘s streams are closely patterned after Kafka\u2019s topics: a stream has multiple partitions, and each partition has an ordered series of values. A stream can have any number of partitions, each of which has a unique key. You can create a stream by pulling data from a topic, but coast<\/code> also has a rich API for building derivative streams: applying transformations, merging streams together, regrouping, aggregating state, or performing joins. Once you\u2019ve defined a stream you like, you can give it a name and publish it out to another topic.<\/p>\n

    By defining streams and networking them together, it\u2019s possible to express arbitrarily-complex dataflow graphs, including cycles and joins. You can use the resulting graphs in multiple ways: print it out as a GraphViz image, unit-test your logic using a simple in-memory implementation, or compile the graph to multiple Samza jobs and run it on a cluster.<\/p>\n

    Sound promising? You might be interested in:<\/p>\n

    Getting Started<\/h2>\n

    The 0.2.0 release is published on Bintray. If you\u2019re using maven, you\u2019ll want to point your pom.xml<\/code> at the repo:<\/p>\n

    \n  bintray-coast\n  https:\/\/dl.bintray.com\/bkirwi\/maven\n\n<\/code><\/pre>\n
    \u2026and add coast<\/code> to your dependencies:<\/p>\n
    \n  com.monovore\n  coast-samza_2.10\n  0.2.0\n\n<\/code><\/pre>\n
    Mutatis mutandis<\/em>, the same goes for SBT and Gradle.<\/p>\n
    Mandatory Word Count Example<\/h2>\n
    val Sentences = Topic[Source, String](\"sentences\")\n\nval WordCounts = Topic[String, Int](\"word-counts\")\n\nval graph = for {\n\n  words  1 }\n      .groupByKey\n  }\n\n  _<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"
    In this dark stream-processing landscape, coast is a ray of light. Why coast? Simple: coast provides a simple streaming model with strong ordering and exactly-once semantics. This straightforward behaviour extends across multiple machines, state aggregations, and even between independent jobs, making it easier to reason about how your entire system behaves. Easy: Streams are built […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4,1],"tags":[],"class_list":["post-7628","post","type-post","status-publish","format-standard","hentry","category-semantic","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts\/7628","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/comments?post=7628"}],"version-history":[{"count":1,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts\/7628\/revisions"}],"predecessor-version":[{"id":8769,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts\/7628\/revisions\/8769"}],"wp:attachment":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/media?parent=7628"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/categories?post=7628"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/tags?post=7628"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}