This is an exported version of the JIRA issue tracker. Please use the Google Code site to open new tickets or report updates to these existing tickets. Feel free to contact the mailing list with any questions.

[NRB-7] A 403 or 401 should be considered a false and not an error
Created: Wed, 5 Oct 2005 08:02:17 -0700 (PDT)  Updated: Wed, 5 Oct 2005 10:34:25 -0700 (PDT)

Status:Open
Project:Norbert
Component/s:
Affects Version/s:0.3.1
Fix Version/s:

Type:BugPriority: Major
Reporter:Henri YandellAssignee:Henri Yandell
Resolution:Unresolved 
Environment:


 Description   
If you can't access a robots.txt for security reasons; the robot should assume the entire site is off limits.
Comment by bayard [ Wed, 5 Oct 2005 10:34:25 -0700 (PDT) ]
Not going to do this in 0.3.2; it's not entirely simple, and would be easier if I became dependent on httpclient (which would make the ssl support easier too). So trying to decide whether I should depend on httpclient.

Given that it may join httpclient at apache someday; not entirely a dumb suggestion.