Improspectives

Title: The Constitution of Algorithms

Author: Florian Jaton

Publisher: MIT Press

Copyright: 2020

ISBN13: 978-0-262-54214-2

Length: 381

Price: $60.00

Disclosure: I received a promotional copy of this book.

There is a vast literature on the process of writing efficient computer programs, but relatively little has been written about the human processes in which those programs are created. In The Constitution of Algorithms, ethnographer Florian Jaton documents his active participation in multi-year project at a Swiss image processing lab to prepare the ground for further research into the human elements of computer programming.

Preparing the Ground

Algorithms, which Jaton loosely defines as computerized methods of calculation, form the backbone of computer programming. These recipes, when properly developed and tested in the image processing context, yield reliable results that compare favorably with human judgment. He breaks the algorithm generation process into three parts: ground-truthing, programming, and formulating.

Ground-truthing is the process of establishing a data set with known correct characteristics. In Jaton’s case, because he joined a group developing face identification (as opposed to facial recognition) technologies, that meant hiring thousands of individuals through Amazon Mechanical Turk to look at a collection of photos and identify the regions, if any, within each image that contained a human face. The team reviewed these evaluations and discarded those that were incorrect. From that base, team members (including Jaton) could engage in programming to create algorithms to identify faces in the photos, which could be compared to the ground truth arrived at earlier. The final section, on formulation, looks at the mathematical underpinnings of these computational techniques. In a real sense the math is the most fundamental aspect of the project, but it wouldn’t make sense to present it earlier because the intended audience of ethnographers wouldn’t have the necessary context to evaluate that information until ground-truthing and programming were described.

The ground-truthing part of machine learning is particularly interesting…one goal of recognition-driven image processing is to identify meaningful, or salient, aspects of a collection of pixels that an algorithm can use to return a true or false value (face or not a face). Salience is tricky – one promising algorithm that distinguished cats from dogs turned out to have been trained on an image set where most of the cats had a collar with a tag and the dogs did not. The algorithm latched onto those tags and, while that criterion worked well for the training set, it failed when applied to other images. I’m also glad that Jaton called out the human effort required to tag thousands of images or perform similar tasks, which is one of the hidden secrets of many machine learning efforts.

Programming as a (Socio)Logical Process

When describing the programming process using a formal system, the author turns to sociotechnical graphs (STGs), which assign a letter to a specific task in a process and track how the tasks enter, move within, and potentially exit a technical process. The author notes that STGs have fallen by the wayside for this type of analysis, and I can see why. While it might be relatively easy for an analyst deeply embedded in a process to keep track of which letter corresponds to which task, doing so will strain a reader’s working memory and make interpreting the STG difficult. I’m not a sociologist and don’t have a recommendation for an alternative system, but I found the STGs hard to read.

What I did enjoy were the Jaton’s interactions with other members of the lab’s team while he developed and corrected an algorithm to generate rectangles that contained faces identified by workers in the Amazon Mechanical Turk program. The common myth of the lonely programmer fueled by caffeine and spite is, thankfully, mostly fiction. Effective programmers seek out advice and assistance, which the author’s colleagues were happy to provide. The lab director took on an outsider with limited coding skills, but Jaton’s willingness and apparent ability to make beneficial technical contributions surely led to friendly and productive interactions.

Conclusion

The Constitution of Algorithms is adapted from Jaton’s doctoral dissertation, which he admits in the foreword was “cumbersome.” There are a few uncommon phrasings and word substitution errors that made it by the editors, but overall Jaton and his MIT Press colleagues did an excellent job of transforming a specialized academic text into a book intended for a broader audience. I believe The Constitution of Algorithms will be useful for sociologists in general, ethnographers in particular, and other analysts who could benefit from a formal approach to the analysis of software development.

Curtis Frye is the editor of Technology and Society Book Reviews. He is the author of more than 30 books, including more than 20 books for Microsoft Press and O’Reilly Media. He has also created more than 80 online training courses for LinkedIn Learning. He received his undergraduate degree in political science from Syracuse University and his MBA from the University of Illinois. In addition to his writing, Curt is a keynote speaker and entertainer. You can find more information about him at http://www.curtisfrye.com and follow him as @curtisfrye on Twitter.