FIRST Robotics Competition Blog

 

We have not yet found a definitive cause for last Thursday’s registration system crash. However, since we restarted the database after the crash, the registration system, which is shared with all other FIRST programs, has stopped exhibiting the anomalous behavior we had recorded prior to the crash. There is a chance the restart solved the issue. Regardless, the database expert we hired will continue to look for the root cause and recommend optimized parameters for us to use when we open registration again. And of course we continue to heavily load test the system under different scenarios.

 

Based on the information we currently have, we are scheduling registration for Wednesday and Thursday next week, with some modifications in approach. While we don’t know for certain that the volume of requests against the system led to the crash, we will be taking significant steps to reduce that volume out of an abundance of caution.

 

Foremost, we will be splitting registration days between District and Regional teams. This change should not impact the likelihood of you being able to register for your desired event, as District and Regional teams won’t be able to Register for each other’s events, and we are using the same first come, first serve method we traditionally have.

 

Initial Registration for District Teams registering for their District events will open at Noon Eastern Time on Wednesday, October 5^th and temporarily close at 9:00AM Eastern Time Thursday, October 6^th,. Registration for District teams will then re-open at Noon Eastern Time on Thursday, October 6^th.

 

Initial Registration for Regional Teams registering for their Regional events will open at Noon Eastern Time on Thursday, October 6^th.

 

(Not sure if you are a District or a Regional team? You will shortly be receiving an email from FIRST with event registration information customized to your status. EDIT: Also check here.)

 

Regional Teams will not be able to register for events during the District Team registration period. However, once the system opens for Regional Teams, it will be open for all, meaning District Teams arriving to the party after Regional Team event registration opens will continue to be able to register for events within their District.

 

Important notes

•  for Regional Teams – from 9:00AM Eastern Time Wednesday, October 5^th, to 9:00AM Eastern Time Thursday, October 6^th, Regional events will not be visible in the system. Events will be visible again starting at 9:00AM Thursday, which gives you time to prepare for the Noon opening.
• for District Teams – as noted above, you will not be able to register for events between 9:00AM Eastern Time Thursday, October 6^th and Noon Eastern Time, Thursday, October 6^th. You will be able to start registering for events again at Noon, when the registration system opens for all.

 

Both of these circumstances are being driven by the system logic we are using to split District and Regional registrations. It is safest and easiest for us to hide Regional events during the District event registration period, then shut down event registration completely for a few hours in preparation for opening Regional event registration.

 

Additionally, we will be shutting down access to external APIs from 11:30AM Eastern Time to 12:30PM Eastern Time on both of these days. These APIs are the way some in our community gather data on the current status of registration to share publicly. We will also be shutting down STIMS and VIMS for this period. We recognize this is an inconvenience, and apologize for that, but we are trying to minimize demand on the system to allow a focus on event registrations themselves. We are also shutting down/minimizing use of internal FIRST HQ systems that would drive additional demand on the system at the same time.

 

This process is only for Initial Event Registration. We anticipate that other registrations will work as usual, without the need for special procedures like this, but precise dates for those Registrations are TBD until we know the system is working. We should be able to release a full set of dates by Friday, October 7th.

 

Also, we have a request. Please do not use bots to attempt to register for your events. While we don’t know for certain, as we said above, that the volume of activity caused the crash last week, we identified several specific teams that were clearly attempting to access the system at a rate higher than a human being sitting at a keyboard could. We were able to identify these specific teams through their IP addresses and the fact that teams must be logged in to the system to register. We had considered building a CAPTCHA speed bump into the process for next week, but recognized that would compromise the experience for all users in an attempt to prevent a handful of teams from adding burden to the system. We will be monitoring registration attempts starting just before event registration opens, and we may be blocking activity from individual teams that are obviously using bots, but we really do not want to take that action, so we are asking: Please don’t use bots. (Yes, we recognize the irony of this request!)

 

We will be continually messaging this information in all traditional channels between now and Registration. You can help by sharing this information with other teams you know, even if you believe they already have the message. We don’t want any team to get left behind next week!

 

Finally, while for several years we have been considering a move to a preferences system for event registration, in which teams could leisurely submit their event preferences over several days, with slots distributed in some fair way at the end of that period, it’s clear the time for that has come. We expect to be moving to an event preference submittal approach for the 2018 season, and we have already begun working on it.

 

I want to thank everyone for sticking with us during this difficult time. The FIRST HQ team is working hard to make this right, and next year, make it better. And it’s still going to be a great season!

Frank