{"id":1454,"date":"2022-08-30T15:16:40","date_gmt":"2022-08-30T15:16:40","guid":{"rendered":"https:\/\/unknownerror.org\/index.php\/2013\/11\/19\/nodejs-deserialization-from-a-stream-collection-of-common-programming-errors\/"},"modified":"2022-08-30T15:16:40","modified_gmt":"2022-08-30T15:16:40","slug":"nodejs-deserialization-from-a-stream-collection-of-common-programming-errors","status":"publish","type":"post","link":"https:\/\/unknownerror.org\/index.php\/2022\/08\/30\/nodejs-deserialization-from-a-stream-collection-of-common-programming-errors\/","title":{"rendered":"NodeJS deserialization from a stream-Collection of common programming errors"},"content":{"rendered":"

I am having an issue deserializing from a stream in node (specifically the pricing feed from the Bitcoin GOX exchange). Basically a chunk arrives which is well formed complete and verified JSON<\/strong>. Here is the code:<\/p>\n

var gox = require('goxstream');\nvar fs = require('fs');\nvar options = {\n    currency: 'AUD',\n    ticker: true,\n    depth: false\n};\n\nvar goxStream = gox.createStream(options);\ngoxStream.on('data', function(chunk) {\n\n    console.log(JSON.parse(chunk));\n});\n<\/code><\/pre>\n
When trying to parse it I get the following<\/p>\n
undefined:0\n\n^\nSyntaxError: Unexpected end of input\n<\/code><\/pre>\n
Any ideas? I have included a sample chunk:<\/p>\n
> {\"channel\": \"eb6aaa11-99d0-4f64-9e8c-1140872a423d\", \"channel_name\":\n> \"ticker.BTCAUD\", \"op\": \"private\", \"origin\": \"broadcast\", \"private\":\n> \"ticker\", \"ticker\": {\n>     \"high\": {\n>         \"value\": \"121.51941\",\n>         \"value_int\": \"12151941\",\n>         \"display\": \"AU$121.51941\",\n>         \"display_short\": \"AU$121.52\",\n>         \"currency\": \"AUD\"\n>     },\n>     \"low\": {\n>         \"value\": \"118.00001\",\n>         \"value_int\": \"11800001\",\n>         \"display\": \"AU$118.00001\",\n>         \"display_short\": \"AU$118.00\",\n>         \"currency\": \"AUD\"\n>     },\n>     \"avg\": {\n>         \"value\": \"119.58084\",\n>         \"value_int\": \"11958084\",\n>         \"display\": \"AU$119.58084\",\n>         \"display_short\": \"AU$119.58\",\n>         \"currency\": \"AUD\"\n>     },\n>     \"vwap\": {\n>         \"value\": \"119.80280\",\n>         \"value_int\": \"11980280\",\n>         \"display\": \"AU$119.80280\",\n>         \"display_short\": \"AU$119.80\",\n>         \"currency\": \"AUD\"\n>     },\n>     \"vol\": {\n>         \"value\": \"249.73550646\",\n>         \"value_int\": \"24973550646\",\n>         \"display\": \"249.73550646\\u00a0BTC\",\n>         \"display_short\": \"249.74\\u00a0BTC\",\n>         \"currency\": \"BTC\"\n>     },\n>     \"last_local\": {\n>         \"value\": \"118.50000\",\n>         \"value_int\": \"11850000\",\n>         \"display\": \"AU$118.50000\",\n>         \"display_short\": \"AU$118.50\",\n>         \"currency\": \"AUD\"\n>     },\n>     \"last_orig\": {\n>         \"value\": \"108.99500\",\n>         \"value_int\": \"10899500\",\n>         \"display\": \"$108.99500\",\n>         \"display_short\": \"$109.00\",\n>         \"currency\": \"USD\"\n>     },\n>     \"last_all\": {\n>         \"value\": \"118.79965\",\n>         \"value_int\": \"11879965\",\n>         \"display\": \"AU$118.79965\",\n>         \"display_short\": \"AU$118.80\",\n>         \"currency\": \"AUD\"\n>     },\n>     \"last\": {\n>         \"value\": \"118.50000\",\n>         \"value_int\": \"11850000\",\n>         \"display\": \"AU$118.50000\",\n>         \"display_short\": \"AU$118.50\",\n>         \"currency\": \"AUD\"\n>     },\n>     \"buy\": {\n>         \"value\": \"118.50000\",\n>         \"value_int\": \"11850000\",\n>         \"display\": \"AU$118.50000\",\n>         \"display_short\": \"AU$118.50\",\n>         \"currency\": \"AUD\"\n>     },\n>     \"sell\": {\n>         \"value\": \"119.99939\",\n>         \"value_int\": \"11999939\",\n>         \"display\": \"AU$119.99939\",\n>         \"display_short\": \"AU$120.00\",\n>         \"currency\": \"AUD\"\n>     },\n>     \"item\": \"BTC\",\n>     \"now\": \"1376715241731341\" }}\n<\/code><\/pre>\n
You can verify it here: http:\/\/jsonlint.com<\/p>\n
Also it is probably worth mentioning I have already tried parsing and removing the escaped characters. Also have tried a couple of different serializers with the same results<\/p>\n
    \n
  1. \n
    You are getting the data chunk by chunk<\/em>. Chunks themselves may not be complete JSON objects. Either buffer all of the data, or use something to do it for you (say the request<\/code> module), or if you need to parse a long stream take a look at the JSONparse<\/code> module.<\/p>\n<\/li>\n<\/ol>\n
    Originally posted 2013-11-19 13:17:31. <\/small><\/p>","protected":false},"excerpt":{"rendered":"
    I am having an issue deserializing from a stream in node (specifically the pricing feed from the Bitcoin GOX exchange). Basically a chunk arrives which is well formed complete and verified JSON. Here is the code: var gox = require(‘goxstream’); var fs = require(‘fs’); var options = { currency: ‘AUD’, ticker: true, depth: false }; […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-1454","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts\/1454","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/comments?post=1454"}],"version-history":[{"count":0,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts\/1454\/revisions"}],"wp:attachment":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/media?parent=1454"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/categories?post=1454"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/tags?post=1454"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}