mini_buildd.httpd module

Error

twisted+SSL: IncompleteRead on event queue

Reproduce: ./devel profile _ssl updatetestall eventually yields to error IncompleteRead(<n> bytes read) (nearly every time).

Error

twisted+SSL: apt-get update form a squeeze platform will stall twisted

squeeze should not really be used nowadays, however no HTTP(S) request should make twisted stall. Still have not broken down what exactly APT@squeeze does differently for https.

Attention

twisted+SSL (mbd_workaround_ssl): Spurious disconnects w/ SSL

WTF: When using SSL, (event queue) connections spuriously disconnect (twisted: “Forcibly timing out client”), even though no timeout has been specified (timeOut=None). Manually disabling ‘abortTimeout’ as well (which otherwise defaults to 15 seconds) seems to fix the SSL case (and hopefully has no other bad effects).

Retests with twisted 22.2 could not (yet) reproduce this behaviour. So I guess it would be fine to remove this once we can pimp the dependency in debian/control.

Retests with twisted 22.4 occasionally also showed (twisted: “Timing out client”), so I added setTimeout(None) as well. I.e., I had no such failures with that workaround, however I also had some successful testsuite runs w/o that workaround.

Attention

twisted < 23.10.0 (mbd_workaround_producer): Twisted can’t serve APT repository when HTTP pipelining is enabled (https://github.com/twisted/twisted/issues/11976)

Avoids seemingly random:

RuntimeError: Cannot register producer <twisted.web.static.NoRangeStaticProducer object at 0xfoo>, because producer <twisted.internet._producer_helpers._PullToPush object at 0xbar> was never unregistered.

errors from twisted. This in turn randomly breaks apt update calls with (slightly misleading) ‘size mismatch’ errors.

This error appears when APT has HTTP pipelining enabled (which it has by default) and reasonable big indices files. You may repeat this bug in plain twisted if you serve any reasonably sized repository like so:

cd <your_repo_dir>
twistd3 --nodaemon --pidfile /tmp/twistd.pid web --path $(pwd)
..
EDIT /etc/apt/sources.list.d/test.list                           # Add resp. APT line
sudo mini-buildd-internals sbuild-setup-blocks apt-clear --run   # Be sure error is not hidden by cached apt lines
sudo apt update
class mini_buildd.httpd.Site(resource, requestFactory=None, *args, **kwargs)

Bases: Site

class mini_buildd.httpd.RootResource(wsgi_resource)

Bases: Resource

Twisted root resource needed to mix static and wsgi resources

getChild(path, request)

Retrieve a ‘child’ resource from me.

Implement this to create dynamic resource generation – resources which are always available may be registered with self.putChild().

This will not be called if the class-level variable ‘isLeaf’ is set in your subclass; instead, the ‘postpath’ attribute of the request will be left as a list of the remaining path elements.

For example, the URL /foo/bar/baz will normally be:

| site.resource.getChild('foo').getChild('bar').getChild('baz').

However, if the resource returned by ‘bar’ has isLeaf set to true, then the getChild call will never be made on it.

Parameters and return value have the same meaning and requirements as those defined by L{IResource.getChildWithDefault}.

mini_buildd.httpd.mbd_is_ssl(request)
class mini_buildd.httpd.FileResource(path: str, defaultType: str = 'text/html', ignoredExts: Sequence[str] = (), registry: Registry | None = None, allowExt: Literal[0] = 0)

Bases: File

Twisted static resource

classmethod mbd_workaround_producer(request)
render(request)

Render a given resource. See L{IResource}’s render method.

I delegate to methods of self with the form ‘render_METHOD’ where METHOD is the HTTP that was used to make the request. Examples: render_GET, render_HEAD, render_POST, and so on. Generally you should implement those methods instead of overriding this one.

render_METHOD methods are expected to return a byte string which will be the rendered page, unless the return value is C{server.NOT_DONE_YET}, in which case it is this class’s responsibility to write the results using C{request.write(data)} and then call C{request.finish()}.

Old code that overrides render() directly is likewise expected to return a byte string or NOT_DONE_YET.

@see: L{IResource.render}

class mini_buildd.httpd.Events

Bases: Resource

classmethod mbd_workaround_ssl(request)
render_GET(request)
class mini_buildd.httpd.HttpD(wsgi_app, minthreads=0, maxthreads=10)

Bases: Thread

shutdown()
mbd_run()