[savannah-help-public] [sr #109544] Intermittent 502 Bad Gateway from gitweb resource

[anonymous]

URL:
<http://savannah.gnu.org/support/?109544>

Summary: Intermittent 502 Bad Gateway from gitweb resource
Project: Savannah Administration
Submitted by: None
Submitted on: Tue 21 Aug 2018 07:41:14 PM UTC
Category: Source code repositories - anonymous access
Priority: 5 - Normal
Severity: 3 - Normal
Status: None
Assigned to: None
Originator Email: ***@pnnl.gov
Operating System: None
Open/Closed: Open
Discussion Lock: Any

_______________________________________________________

Details:

We run a matrix build on Travis CI (https://travis-ci.org/GlobalArrays/ga)
that downloads the config.sub and config.guess resources via wget.

For example:

wget -O config.guess
'http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD'

We're getting intermittent 502 Bad Gateway HTTP codes that cause our builds to
fail.

I'm trying to find out if you may have a suggestion on a better & more
reliable way to download these resources?

Since the build periodically passes and fails without changing the scripts, I
suspect our build combined with the rest of the requests your servers receive
is overloading the system? This is just a guess, so there may be a better
explanation.

Thanks for the help!




_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/support/?109544>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/

[Bob Proulx]

Follow-up Comment #1, sr #109544 (project administration):

For about a week there starting around the 14th and continuing through the
19th we were experiencing about a 10,000 botnet DDOS attacking the vcs system
browsing every web accessible URL. This has caused a big stress on the
system. Lots of cpu use, lots of data bandwidth consumed, but mostly lots of
processes queued. Users have made reports of timeouts and 502 Bad Gateway
proxy failures and other brown-outs while browsing the web or checking out
project source. No hard failures. Retrying the action would usually work.
But also the failure would occur again too.

Unfortunately mitigating against a large botnet attacker with a large
collection of systems to do the attack is quite difficult. Even very large
and well funded sites have been brought down by these attacks. For us on the
community funded FSF hosted Savannah these can be overpowering and there isn't
much we can do about it. We took what mitigations we could.

The current attack subsided around the 19th. For the moment everything seems
to be normal. Let's hope that continues.

Are you subscribed to the savannah-users mailing list? Please consider
subscribing to it. It is low traffic volume.


_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/support/?109544>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/