Tag Archives: mozfest

Hacking Mozfest Wrangling

For those unaware, for the past 3 years now I have been involved in the Mozilla Festival over in London, specifically with the YouthZone. In 2016 and 2017 I have been a “Wrangler” for the space. This role involves selecting sessions from submissions, working with them over the next few months and finally then putting it all together for a weekend at the end of October.

This year at Mozfest, due to a number of different factors, I ended up taking on a lot more of the administration for Youth Zone than previous years.
This post is an overview of how we hacked away at this process, automating as much of it as possible.

Overall, there is a lot of administrative work loaded onto wranglers of the festival. In our case, 60 sessions with a combined total of 140+ facilitators made everything from communication to scheduling a target for automation.

Where we started

Before getting into how I ended up hacking the process further, lets first look at some of the underlying building blocks of curating the festival.

Overall, over 800 sessions were put forward. To put a session forward, a potential faciliator simply fills in a form on the Mozfest website asking plenty of different details for their session. Once they hit submit, a public Github Issue is created and the facilitator is emailed with details of how they can use this system.

It is a great way of making use of Github’s system as it provides a mechanism for having a public conversation with the potential facilitator using the comments, while also using the extremely powerful milestone/label system to categorise the sessions.

As well as the public Github issue, there is also a row in the master spreadsheet in behind that includes the private information like the facilitators email address etc. This spreadsheet can only be accessed by wranglers and the production team.

So at this stage, we are here, a Github repository full of issues, each with a milestone (the primary session it has been submitted to) and some labels (including secondary space, language etc), with a spreadsheet in behind with the other information.

 

Selection

The next step for myself and my fellow wranglers was selecting the sessions we liked the sound of to accept, in general we finished most of this by the start of September. To mark sessions we had accepted, we simply created a label and attached this to each of the sessions we were accepting.

Information collection

Once we knew the sessions we wanted to accept, we reached out to each primary facilitator with an email asking them to fill in the additional YouthZone admin form. This form included a number of questions extra including which days they could make it to the festival to run their session, DBS checks (UK police background checks) and if travel/accommodation assistance was required.

For all our forms, we used Google forms given it would feed directly into Google Sheets.

On top of this admin form, throughout the process we also had additional forms for DBS information, pizza preferences and accommodation parents details.

Alongside these Google forms, we also used a custom Google Script based form for uploading of files like worksheets/resources. Each item submitted adds a row to a Google Sheet, while also uploading the file itself to a Google Drive folder and adding a direct link to the spreadsheet.

One of our custom Google Script based forms for uploading worksheets to be printed.

Scheduling

Once the selection was complete, it was time to look at scheduling. At this point (sometime in September), we weren’t aware of Guidebook (the system used in the end for the schedule app at the festival), but wanted to get started with scheduling sessions.

In Youth Zone, we have 10 individual areas/rooms that sessions can run in and 11 available time slots. In previous years, Dorine (my co-wrangler) had scheduled simply using Post-It-Notes on a whiteboard while I had used tables in Google Docs with links back to the Github Issues. Although both these methods worked, they were far from ideal… Both worked fine for the initial scheduling, but feel apart if any major changes needed made.

So I proposed an experiment to Dorine, what about trying Github Projects?
For those unaware, Github Projects is a Kanban Board style system that allows columns where either issues/pull requests or “cards” can be placed. The huge advantage over other options like Trello though is as it is a Github system, it understands the initial Github Issues themselves, of which each session is one of.

We were able to use pretty complex search filters to find the required Issues and place them into room columns on the board. A perfect feature in Github Projects for us was that an Issue is removed from available Issues if it is added to the board, meaning we could make sure we had every session scheduled somewhere. For repeating sessions, we simply added the links to the Issues into cards.

Duplicate sessions could be added easily by just adding the URL into a text card, Github then showed the referenced issue below the card.

Next step though was as Issues/Cards have different widths based on the number of labels etc, I put together a simple Flask based app to pull the data from Github and display it in an HTML table.

As can be seen in this image, the cards themselves don’t line up across, so another solution was needed to render them.

Although building the Flask app was pretty simple, actually getting the data out of Github was another story.
Github have 2 APIs, version 3 (REST based) and version 4 (GraphQL based). After a recommendation from the fantastic Joe, I went for version 4 using GraphQL.

GraphQL is a really interesting way of doing an API, it allows you to ask for lots of different bits of information in a tiered way using a single API call.

At first though, although the amount of Github documentation for the GraphQL API is large, there is little documentation on how to implement it outside of the GraphQL explorer. For someone with little API querying experience in general, I found figuring out how to move my query over to Python challenging. In the end though, after a few evenings of tinkering I came across the required parameters/formatting needed to get requests.post to work with my query. An example using this written in Python 3 is below.

So next, I needed to get the actual information from the Github Project so I could display it in a more visually useful way.

The result was the following query

{
organization(login: "MozillaFoundation") {
  repository(name: "mozfest-program-2017") {
    project(number: 1) {
      columns(first: 30) {
        edges {
          node {
            id
            name
            cards(first: 10) {
              nodes {
                id
                note
                content {
                  ... on Issue {
                    id
                    title
                    url
                    body
                    labels(first:20){
                      nodes{
                        id
                        name
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}
}

The query pulls back from the mozfest-program-2017 repository, our project (number 1), followed by all the columns, then all the cards in each column, then if the card is actually an issue (the … on Issue syntax), also include some more details and any labels the issue has attached to it.

Although perhaps a little overkill, that query returns over 100,000 characters (mainly because it includes the descriptions). This though is something that would have required many REST API calls, but can instead be done with a single GraphQL call.

The result is then the results from the query are fed into a simple and pretty rough looking HTML table.

Once I had all the data from the API, it was then as simple as firing up a VM in the cloud with a Flask app displaying the data in a pretty rough looking HTML table. Although I have no doubt this could have looked a heck of a lot nicer with some more CSS, it served the needed purpose of showing all the data is rows/columns, allowing us to build a rough schedule.

Catching mistakes

The next progression for this simple Flask app was using it to catch mistakes.
With 60 sessions, it is very easy to make simple mistakes like scheduling a session on the Sunday, when the facilitator can only make the Saturday for example.

To combat this, I put together a separate piece of Python code that went through each response to the YouthZone admin form used the URL to the session provided and added a label on Github of either Saturday only, Sunday only, either but only one day and either/both days.
Unfortunately there is no GraphQL API call for adding labels to issues just yet, so this one had to be done using the old v3 REST API.

This now meant that nearly all sessions now had a “day they could run” label attached. Although this was then easy enough to check the label and schedule accordingly, it could be taken a step further so I added a little more Python code to the Flask app to change colour based on if scheduling mistakes were made. For example, if a facilitator said they could only make it on Saturday and we scheduled their session on Sunday by mistake, the session would flag up orange. This saved us from 2-3 mistakes at least, which would have involved very last minute changes if we hadn’t caught them so early.

Guidebook

By the end of September, we had a rough schedule ready to go and the Mozfest production team informed wranglers the plan was to use a new 3rd party system this year, Guidebook.

Guidebook is a well thought out system for building festival programmes, venue guides etc with a web based builder and auto updating IOS/Android apps along with a web app as well.

Once we had access to Guidebook, I started looking for an API to directly interact with it. Unfortunately at the time of writing this, there is still no API for Guidebook. This presented an issue for us as we had all our scheduling information in a Github Project which had systems querying it already thanks to Githubs great API. So we had 2 options

  1. Import all of our session data into Guidebook, manually add the times/locations to each session and kill off the Github Project. This though means nothing else can access the schedule data outside of Guidebook itself.
  2. Work out some other way to keep our primary source of “truth” outside of Guidebook and sync it in (given there was no way to get the data out of Guidebook).

I decided to dig into option 2 and after chatted with the excellent Guidebook support team, we concluded the built in calendar sync tool might be an option.

Guidebook includes an option to sync sessions from a calender feed (ICS format), then populate these sessions into a selected track and do this every 2 hours.
This brought with it a few problems though

  1. We could no longer use the built in system for sharing a link from Guidebook to a facilitator to allow them to edit their session details (title, description etc) themselves.
  2. We couldn’t use some of the more fancy Guidebook features like attaching a facilitator bio to a session or adding files.

Editing session details

Guidebook’s built in system for allowing facilitators to edit their session details. This needed replicated over to our system if we wanted to work outside of Guidebook.

To make this calender option at least a viable solution, we needed a way to allow facilitators to edit their session details themselves, as in a majority of cases their session had slightly changed one way or another since they submitted it. For each of the other spaces, they used the built in collaboration link in each Guidebook session to send that link out to the facilitators involved in a session so they could edit it themselves.

My answer to this? Simply build a copy (with a little less CSS again).

Primary facilitators could use the above form for their session to edit the title/description.

I added some random UIDs to the end of a copy of the overall session submission spreadsheet and put together a quick Python 3 script using the Google Sheets API (and library) to email out to each primary facilitator this magic link that would allow them to edit their session.
When a user of the web system enters a URL that matches a set pattern, the Python code goes and searches for that UID in the spreadsheet. If it finds it, it returns a page with all the information ready to be edited. If it doesn’t find it, it simply returns an error page.

Building a calendar

The final step needed for getting our schedule in Github Pages over to Guidebook was building a calendar for Guidebook to keep an eye on. For this, we used Google Calender and the Google Calender API.

Every so often in the process leading up to the festival, I would run this script that would update any sessions that had had details (times, location, title, description etc) changed. Guidebook would then pull in these changes every 2 hours (or immediately if manually fiddled with).

The auto generated calendar feed for Saturday with each session, along with its location.

Out solution at this stage worked, but had some issues.

  1. All the columns from the Github Project were being used to generate calendar entries, including the columns with time slot information and some columns that were just for information.
  2. The system had no answer for all day sessions or sessions outside those fixed time blocks.

The answer to both problems? A set of symbols to include in the column titles on Github Projects.

Note the symbols added to the columns, for example an # signifies this is a time block column and a @ signifies ignore this column.

As well as using # and @, anything inside brackets was ignored for room details and simply left as a note for us.

As well as those symbols, we also added all day long sessions into a column marked with (!all-day).

Overall, this allowed us to schedule sessions for a full day, along with include more information in the column title (for us when planning), but still have it pull the exact room name from the column title.

Volunteering

The final piece in our admin puzzle was making sure we had enough people helping out in each session. This was mainly important for the hands on workshops etc, as they needed plenty of helpers at the side.

In previous years, facilitators just put their names down in a separate column in the Google doc. This year though I was keen to find a better and more automated solution.
The answer in the end was building a system that sat on top of the schedule viewer. Like with the editing session details section, I added a unique ID into another column in our admin form spreadsheet, then emailed that unique edit link out to everyone. This unique link allows them “logged in” access to the volunteer signup page.

Each volunteer was sent a custom unique link to the volunteer system.

The volunteer system allowed facilitators to click on a session they wanted to help with, it would then change to blue to signify they were signed up to help at it. The session blocks then changed colour based on how much they were still in need of people to help with.

The result of this was we knew on Friday afternoon / Saturday morning which sessions didn’t already have people offering to help at them and could ask facilitators who hadn’t used the system, to help in those specific sessions.

As well as signing up, facilitators could also click “show info” on each session and see who was already signed up to help.

 

To conclude

Organising a festival is hard. It takes a team of people a huge amount of work going on behind the scenes to make it all run like clockwork. For a festival the scale of Mozfest with 2000 attendees and 450 facilitators overall, automation is critical to avoid the team burning out, especially when a majority of the Wranglers involved with the festival are volunteers.

I am a big fan of the openness around the Mozilla Festival using Github as the base for the information initially. I think it is a very creative idea that in practice does actually work well.
The fact that the entire selection process for the festival is done in public and in the open is something I have not seen many other conferences do and is something I feel they could learn from.

Is the process perfect? No, by no means! But the availability of the excellent Github API allows for the process to be extended even further than it currently is. With YouthZone, we were only reaching the tip of the iceberg of what could be done to automate the processes of the festival.

So I hope the breakdown of how we “hacked” the running of the festival this year, can serve as a springboard for building better tools and processes for next years festival, to avoid the hundreds of hours wasted unnecessary each year by Wranglers on repetitive and easy-to-make-mistakes-in tasks.
All this to allow the Wranglers to do what they do best, wrangler their spaces!

Mozfest 2016

This year once again, I was involved with the Mozilla Festival on the final weekend of October 2016. This year, I was (somehow) persuaded to be a Space Wrangler, alongside Dorine Flies.

Berlin

The planning process for us started back in May, when along with the other Space Wranglers and Mozilla staff, we traveled to Berlin for a long weekend planning retreat.

While in Berlin, we planned out how we wanted the YouthZone to pan out and started discussing the types of sessions we would love to nab for the space. We also had many interesting conversations with other Wranglers on possible collaborations at the festival.

Festival lead up

Over the next few months in the lead up to the festival, the call for proposals opened and we received well over 100 session proposals. We had painful task of then whittling those down to just over 60 sessions for the weekend. This entire process was done in the open online via the festival proposals Github repository.

Friday

The Friday of the festival was the setup and facilitator training day.
At it, we had about 45+ of our fantastic YouthZone facilitators.

The first part of the morning was facilitator training, provided by Mozilla.
After this, all the spaces split off to their respective floors. We got everyone from the YouthZone together and went over specific details for the weekend.

After lunch, we started the logistical nightmare of setting up 64 Raspberry Pis across 2 Pi labs!

Vincent busy doing the initial equipment inventory.
Vincent busy doing the initial equipment inventory.
So many PiTops!
So many PiTops!

The setup process, although a very complex one, was completed in a few hours thanks to help from the large army of volunteers.

The result was a main classroom featuring 38 PiTop Ceeds and a smaller classroom featuring 26 DVI monitors (from Computer Aid International). The entire setup made use of Raspberry Pi 3s (provided by PiTop and Raspberry Pi Foundation), a wired network (with a little help from Ravensbourne and NI Raspberry Jam) and to top it all off, running the next experimental PiNet Jessie.

Bawar running final tests on PiTops
Bawar running final tests on PiTops

Along with setup, some of the team spent the afternoon creating the posters to plaster around the Ravensborune.

We also had the laser team (headed up by Amy Mather) upstairs on the 9th floor getting their lasercutting workshops ready for the weekend.

Saturday

The festival officially opened on Saturday morning, with the team down for 8am (something that didn’t go down very well earlier when they were told they needed to be down by then).

Throughout the day, the Raspberry Pi volunteer team ran 12 workshops from Dave Hones’s “Code in Space” workshop to 12 year old Elise’s “Spooktacular Sonic Pi” workshop and Femi/Nic’s fantastic Crumble robot workshop!

Frank-Pi Pi-group

 

 

In parallel to the Pi workshops, we also had Cat Dunicliff running her VR workshops using kit on loan from Google Expeditions team.

Cat-expeditions

 

Then, up on the 9th floor, throughout Saturday, Amy Mather and her team ran a series of lasercutting beginners workshops.

 

Laser-example Amy-laser-kid Amy-laser-laughing

On the Saturday evening, 35 celebrated the success over pizza! Unfortunately due to a minor mixup with the restaurant, we ended up split across 2 of the restaurants across London. Will say though, Franco Mancas pizza was excellent.

"My pizza has arrived!" - Cerys (with slightly tired Amy)
“My pizza has arrived!” – Cerys
(with slightly tired Amy)

Pizza-2 Pizza3

Sunday

On the Sunday morning, the team had a little longer of a lie in, with them all arriving for 9am.

Once again, we had another fantastic set of workshops with Raspberry Pi based workshops ranging from Edublocks by Josh, Scratching stuff from your kitchen by Aoibheann and automatic twitter powered photobooth by Vincent/Sam.

Amy and the lasercutting team continued another 2 lasercutting workshops on the Sunday and Cat was joined by Pietro from Google to run some Google Expeditions workshops.

Josh-edublocks

Conclusion

Mozfest has been a fantastic journey, starting right back in May, up to now. We have already started discussions for 2017.

Myself and Dorine set out from the start with the mission to provide youth (and the young at heart) the opportunity to learn new skills and discover exciting new technology. We set out to widen what the attendees (especially kids) thought was possible and provide them with opportunities to see that hands on.
These are going to be the digital innovators of tomorrow!

Alongside this mission, we had a 2nd, less shouted about mission. We wanted to provide opportunities and build up the skills/confidence of our amazing volunteer team. Over half of the sessions run at YouthZone were lead by youth. Our youngest facilitator this year at Mozfest was only 10! Of those youth, a majority had never given a proper workshop to a group of complete strangers.
We gave them this opportunity in a safe space, always with someone with a little more experience on hand, just in case. The feedback from them all has been amazing and I am pretty confident we will have over 90% of this years team, applying again to run sessions next year.

Personally, I feel we accomplished both of these over the weekend. There are of course numerous things we could have done better, but on the whole, I think we didn’t do half badly.

Thanks

To pull off 25 Raspberry Pi based workshops, 4 Google Cardboard workshops and 5 lasercutting workshops, required a heck of a lot of help.

First, I want to say a huge thanks to the army of volunteers that got thrown together over the past few months. Although they all already know it, it is worth saying again… You guys were all awesome! Well over 45 of them showed up over the weekend and were given jobs ranging from lifting monitors, to wrapping cables to counting the exact number of crocodile clips at the end to creating 30+ posters!

The other big thanks I want to make is to our fantastic partners. They provided equipment and staff to make whole thing logistically possible. These include (in no particular order)

Andrew-Femi