My experience with Flock 2017

After attending Flock 2016, I got another chance to be part of Flock conference. This year, it took place in beautiful city Hyannis, Massachusetts, USA from 29th August to 1st September. Schedule of this 4 day conference was designed differently compared to last year. Both workshops and talks were running in parallel for the first three 3 days followed by a wrap-up session on last day.

Flock has lots of talks/workshops that are full of hot topics which are currently happening in Fedora. Being part of Fedora Alternative Architectures group, it is a good place for me to learn and understand about what is new in mainline Fedora. Also, I get to discuss with people to get an overall idea on efforts required to get those features available on different architectures like PowerPC, s390x, ARM.

This year, I attended various talks and workshops on topics related to Project Atomic, Modularity and Fedora CI. Other than that, I co-presented a workshop on Alternative Arches debugging and fixing with Dan Horak.


Day 1 of the conference started with keynote from our Fedora Project Leader, Matthew Miller . Best part of the talk was lots of graphs related to Fedora progress in various aspects. These included number of users using various latest Fedora releases, Fedora contributors activities across time, etc. He highlighted Fedora Atomic CI and Modularity as one of the two current objectives for Fedora.



Some of the interesting talks I attended were:

Factory 2.0

In this talk Mike Bonnet talked about development around several tools to make a better and stable fedora:

  • ResultsDB – gets info from autocloud, openqa and taskotron
  • WaiverDB – waiving a failed result, when we know test is failing incorrectly
  • Greenwave – correlates data from ResultsDB and WaiverDB
  • bodhi – enhancement done to incorporate Greenwave result
  • Freshmaker – triggers rebuilds of content based on events from other services in pipeline like rebuild a module when modulemd or specific file gets updated, rebuild containers when rpm gets to stable
  • Odcs – On Demand Compose Service generate repos (signed) of pre-release content for inclusion when building aggregate content(like: ostree, etc)

Multi-Arch Container Layered Image Build System

In this talk Adam Miller covered various topics like differences between base and layered container image. In Fedora, base image gets built by Fedora releng and layered image via OpenShift Build Service (OSBS). Currently, container image gets built only for x86_64 architecture. There is a new multi-arch build-system design in progress, which will have OSBS workers for multiple arches. Multi-arch images will further be available in registry which can be easily accessed by users.


New Container Technologies

Another nice talk from Dan Walsh on containers with the idea that “a container should be as generic as pdf ”. He introduced various tools being developed to achieve building generic container images. Some of the tools are:

  • skopeo  – various operations on container images and image repositories
  • Buildah – to create image without docker file. Has
  • cri- o – A daemon to ensure that kubernetes doesn’t break by providing end to end test. Later also integrated with openshift test suite
  • Kpod –  alternative of docker-cli

Automate Building Custom Atomic Host with Ansible

In this talk, Trishna Guha explained how to build your own custom OSTree and rebase an Atomic Host system with built custom OSTree. First approach involved doing all steps manually which is really tedious work. Later she demonstrated power of Ansible by adding steps involved in ansible playbook. I love ansible and its power!


Alternative Arches debugging and fixing

This session was organized by Dan Horák and me.  During this session, Dan talked about common architectures specific issues which may occur like data endianess, usage of signed/unsigned char, etc. He also talked about various resources and links available for people to help with debugging non x86_64 specific issues. One can get access to a non x86_64 machine available at Fedora wiki. Further talked about recent work started towards CI to track changes in upstream by regularly building them on required arches.


Later, I did a short walk through on an issues which I am debugging from quite sometime. This issue involves storage configuration error during anaconda install while building Atomic CloudImage locally on armhfp. It involves looking into Python code for error, followed by C Python binding code where actual problem may lie. This example was chosen to show how deep down an arch specific problem can lie and how we should step in to debug them patiently.

We further planned to work on fixing arch-specific issues which attendees might have in their packages but there were none from available attendees. Interested folks can look into slides and notes on debugging issue example and can talk to us later on IRC/ML.

Atomic Host 101

This session was taken by Dusty mabe which started with introduction on Atomic Host and followed by hands-on. Dusty did great job in distributing workshop’s prerequisites (Fedora Atomic vagrant image and local ostree repo) in advance. During the workshop, we did the following tasks which is nicely written in series of blogpost by Dusty:

I got a better understanding of Atomic Host from from this workshop. Initially faced some problems related to storage expansion of Atomic Host in vm, which got fixed with the help of Dusty.

Fedora Atomic Doc Work

This session was organized by Josh Berkus and Trishna Guha. Being interested and involved in Atomic Project, I wanted to attend this session in order to help with documentation as well. Josh and Trishna gave an overview of how these docs are organized and how to build and start adding content  here. Atomic Docs content are written in AsciiDoc and uses AsciiBinder to build and maintain documentation. Firstly, I did initial setup to build the Atomic Host doc repo locally and then picked up an issue from list which was about OSTree Compose. This first requires understanding of how OSTree compose is done, trying out local steps and then write the doc. Due to the nature of the topic, I promised Josh to work on it properly after Flock. At the end of the workshop, got a hot and spicy prize from Josh (thanks!)


Let’s create a module

This session was taken by Tomáš Tomeček on creating a module. During this session he took us through content of modulemd files. Modulemd is a yaml file which has various fields defined from the list specified here . To build your own module locally, write a modulemd spec file and then run mbs-build local. This seems very simple but it takes time to do initial setup and getting successful build done.

Wrap-up session

During last day of the conference, various attendees shared their experience on what they learnt or got done during last 3 days. Speakers talked briefly on how their session was received by attendees. This session was nice but I would prefer having a wrap-up session at the end of each day of conference rather than on last day.

What else?

Other than attending various sessions, I also took opportunity to talk with some of the people regarding problems I came across on multi-arch:

  • Talked with Adam Miller regarding status of building layered images on multi-arches. Layered images get build using OSBS which currently supports x86_64. Thanks to Adam on continuing his work to support building images for other arches as well.
  • Also, discussed with Adam Miller about possibilities on adding non x86_64 arches container images in Fedora registry. Right now, only x86_64 is possible. We had a short discussion along with Randy Barlow and it seems that it should be possible after adding a manifest file. I will be looking into video provided by Randy on scaled container registry and will work with him to make Fedora container registry mult-arch aware.
  • User Feedback on Modularity :  I also went to Modularity feedback session to see what’s going on here. I previously gave it a try when Boltron – Modular server preview release occurred. I was more interested in terms of getting it working on multi-arches. Discussed issues I came across while while trying out on ppc64le:
    • Fedora 26-modular iso boots and starts anaconda installer in text mode. Didn’t see a way to start in graphical mode.
    • Fedora 26-modular iso is not able to fetch nearest mirror during anconda installation. Even proving manual repo path, doesn’t seem to work.

Modularity team took these feedback positively and I hope that these issues will get addressed in upcoming release.

Also, I asked for possibility of getting Modular Fedora container image built for multi-arches to try out. Right now, it’s not possible because it gets built using osbs (multi-arch build support is in progress). Good thing is Tomáš Tomeček volunteered to help me with building container image locally. He will first try out on x86_64 and will share a blogpost on same.

I am looking forward to spend more time on Fedora-modular on multi-arch.

There were many more interesting sessions but I couldn’t attend due to multiple sessions in parallel. Waiting for videos from talks and workshops to be uploaded in order to watch some of them which I missed.

I fully enjoyed Flock this year – from attending sessions to meeting Fedora friends to evening Flock events to winning lot of goodies at Wackenhammer’s Clockwork Arcade and Carousel 😉 Thanks to Brian Exelbierd, rest of the organizers, volunteers and sponsors for making Flock awesome. Also, big thanks to my employer Red Hat for sponsoring my trip!

It was nice to meet you all in person, now it’s time to go back and continue working on things on fire until we meet next time 🙂