{"id":2451,"date":"2022-08-30T15:24:58","date_gmt":"2022-08-30T15:24:58","guid":{"rendered":"https:\/\/unknownerror.org\/index.php\/2014\/01\/12\/cowboy-web-server-application-very-slow-collection-of-common-programming-errors\/"},"modified":"2022-08-30T15:24:58","modified_gmt":"2022-08-30T15:24:58","slug":"cowboy-web-server-application-very-slow-collection-of-common-programming-errors","status":"publish","type":"post","link":"https:\/\/unknownerror.org\/index.php\/2022\/08\/30\/cowboy-web-server-application-very-slow-collection-of-common-programming-errors\/","title":{"rendered":"Cowboy web server application very slow-Collection of common programming errors"},"content":{"rendered":"

I am currently playing around with minimal web servers, like Cowboy. I want to pass a number in the URL, load lines of a file, sort these lines and print the element in the middle to test IO and sorting. So the code loads the path like \/123, makes a padded “00123” out of the number, loads the file “input00123.txt” and sorts its content and then returns something like “input00123.txt 0.50000”.<\/p>\n

At the sime time I have a test tool which makes 50 simultaneous requests, where only 2 get answered, the rest times out.<\/p>\n

My handler looks like the following:<\/p>\n

-module(toppage_handler).  \n-export([init\/3]).   \n-export([handle\/2]).   \n-export([terminate\/3]).    \n\ninit(_Transport, Req, []) ->\n    {ok, Req, undefined}.\n\nreadlines(FileName) ->\n    {ok, Device} = file:open(FileName, [read]),\n    get_all_lines(Device, []).\n\nget_all_lines(Device, Accum) ->\n   case io:get_line(Device, \"\") of\n       eof  -> file:close(Device), Accum;\n       Line -> get_all_lines(Device, Accum ++ [Line])\n   end.\n\nhandle(Req, State) ->\n    {PathBin, _} = cowboy_req:path(Req),\n    case PathBin of\n        -> Output = ;\n       _ -> PathNum = string:substr(binary_to_list(PathBin),2),\n            Num = string:right(PathNum, 5, $0),\n            Filename = string:concat(\"input\",string:concat(Num, \".txt\")),\n            Filepath = string:concat(\"..\/data\/\",Filename),\n            SortedLines = lists:sort(readlines(Filepath)),\n            MiddleIndex = erlang:trunc(length(SortedLines)\/2),\n            MiddleElement = lists:nth(MiddleIndex, SortedLines),\n            Output = iolist_to_binary(io_lib:format(\"~s\\t~s\",[Filename,MiddleElement]))\n    end,\n    {ok, ReqRes} = cowboy_req:reply(200, [], Output, Req),\n    {ok, ReqRes, State}.\n\nterminate(_Reason, _Req, _State) ->\n    ok.\n<\/code><\/pre>\n
I am running this on Windows to compare it with .NET. Is there anything to make this more performant, like running the sorting\/IO in threads or how can I improve it? Running with cygwin didn’t change the result a lot, I got about 5-6 requests answered.<\/p>\n
Thanks in advance!<\/p>\n
    \n
  1. \n
    The most glaring issue: get_all_lines<\/code> is O(N^2) because list concatenation (++<\/code>) is O(N). Erlang list type is a singly linked list. The typical approach here is to use “cons” operator, appending to the head of the list, and reverse accumulator at the end:<\/p>\n
    get_all_lines(Device, Accum) ->\n   case io:get_line(Device, \"\") of\n       eof  -> file:close(Device), lists:reverse(Accum);\n       Line -> get_all_lines(Device, [Line | Accum])\n   end.\n<\/code><\/pre>\n
    Pass binary<\/code> flag to file:open<\/code> to use binaries instead of strings (which are just lists of characters in Erlang), they are much more memory and CPU-friendly.<\/p>\n<\/li>\n<\/ol>\n
    Originally posted 2014-01-12 20:50:59. <\/small><\/p>","protected":false},"excerpt":{"rendered":"
    I am currently playing around with minimal web servers, like Cowboy. I want to pass a number in the URL, load lines of a file, sort these lines and print the element in the middle to test IO and sorting. So the code loads the path like \/123, makes a padded “00123” out of the […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-2451","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts\/2451","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/comments?post=2451"}],"version-history":[{"count":0,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts\/2451\/revisions"}],"wp:attachment":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/media?parent=2451"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/categories?post=2451"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/tags?post=2451"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}