Error on accessing via ?url= query parameter

[premasagar]

In the develop-cheerio branch.

Request:
GET http://localhost:8888/?url=chrisnewtn.com

Console:

info: -----------------------------------------------
info: Elsewhere started - with url: chrisnewtn.com
info: parsing: chrisnewtn.com
warn: TypeError: Cannot read property 'statusCode' of undefined
info: total pages 1
info: total fetched 0
info: total errors 0
info: total pages outside limits 0
info: total html request time: 0ms
info: total time taken: 4ms

Response:

{
"results": [],
"query": "chrisnewtn.com",
"created": "2012-10-12T23:04:46.411Z",
"crawled": 1,
"verified": true
}

[premasagar]

Ah, no, it turns out the issue is that the http:// protocol was missing. The same happens when passing in chrisnewtn.com without the protocol in the select box in the demo.

[glennjones]

I took out the code for dealing with missing http:// protocol as it only in
the bin/elsewhere server code not the lib. I was having problems testing
the lib when the html interface acted differently to using the
code directly. It would actual more useful if the html interface had all
the lib options in a form so that we could play with them.

I am getting to the point where not having a unit test system is a problem.
Just added some code that now has a cause a another problem with badly
formed URL structures.

I will try and fix it.

Glenn

On 13 October 2012 11:10, Premasagar Rose [email protected] wrote:

Ah, no, it turns out the issue is that the http:// protocol was missing.
The same happens when passing in chrisnewtn.com without the protocol in
the select box in the demo.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/45#issuecomment-9402709.

[premasagar]

Ah, I see. Yes, the demo should work the same as when using directly
as a module. Yes, form controls for the options would be great so
that, rather than just a demo page, it becomes an API explorer.

I do like http:// being considered the default protocol, so that it
can be optionally omitted but yes, as you say, the behaviour needs to
be consistent, wherever the module is used.

On that, I'm also aware that the demo page has "strict" checked, but
the default global option is strict=false. Should we change the
default global option so that strict is true? (Or vice versa?)

Yes, we should have tests. We've used mocha + chai + sinon in other projects.