How I Ruined Office Productivity with a Face-Replacing Slack Bot

Chris works in an office full of people that like to Photoshop his face onto various things, and then post the results in the company Slack.

But opening Photoshop and copy/pasting face cutouts can be tedious, especially when Chris is trying to distract you with tales of his Smite heroics. After weeks of long nights in Photoshop I was determined to find a better way, and so the idea for @Chrisbot was born.

When I first conceived of the idea, I knew there would be three major components to the project:

1. Simple image manipulation
2. Slack integration
3. Face detection

I had browsed the Go image and image/draw packages in the past, and read a few articles on them, and I was confident that I would be able to use it for the purpose I had in mind. That took care of component number 1.

I had made a toy Slack bot in Go in the past, following some instructions I had found on Google. The lack of official/comprehensive Go Slack client made it tricky, but for basic needs I was confident I could make a bot capable of downloading and uploading images via Slack. That took care of component number 2.

The only piece I was unsure of was how easy it would be to detect faces. I Googled golang face detect and clicked the first result, a StackOverflow question about the go-opencv computer vision library. A quick look at the face detect sample project in said library told me everything I’d need to know. Component number 3 was as good as solved.

Face Detection

Being the most unfamiliar piece, I started out with the face detection. It was the biggest unknown in the project, and it would make little sense to create the rest if I couldn’t even figure out where to paste the faces.

I decided to encapsulate the go-opencv library as best I could. I could tell that the opencv data types were dissimilar to the Go standard library, at least in how it defines the Image and Rectangle interfaces, so some conversions would be necessary.

With a bit of digging I found a reference to the opencv.FromImage method, which would handle the conversion from Go’s image.Image to the opencv library’s. This had the added benefit of no longer requiring that I pass a file path to the opencv.LoadImage method, meaning I could instead work with an image stored in memory. This would potentially save me the step of saving the image to the filesystem after receiving it from Slack.

Unfortunately I was unable to find a way to provide the same convenience for loading the Haar face classification XML file, but being impatient I decided that was something I could live with.

With that, I was able to make something akin to the below facefinder package, which looked far more rough a few iterations ago:

package facefinder

import (
  "image"

  "github.com/lazywei/go-opencv/opencv"
)

var faceCascade *opencv.HaarCascade

type Finder struct {
  cascade *opencv.HaarCascade
}

func NewFinder(xml string) *Finder {
  return &Finder{
    cascade: opencv.LoadHaarClassifierCascade(xml),
  }
}

func (f *Finder) Detect(i image.Image) []image.Rectangle {
  var output []image.Rectangle

  faces := f.cascade.DetectObjects(opencv.FromImage(i))
  for _, face := range faces {
    output = append(output, image.Rectangle{
      image.Point{face.X(), face.Y()},
      image.Point{face.X() + face.Width(), face.Y() + face.Height()},
    })
  }

  return output
}

With this, I could find faces as easily as:

imageReader, _ := os.Open(imageFile)
baseImage, _, _ := image.Decode(imageReader)

finder := facefinder.NewFinder(haarCascadeFilepath)
faces := finder.Detect(baseImage)

for _, face := range faces {
  // [...]
}

I threw in some copy/paste “draw a rectangle” code I found on Google to sanity check and confirm it was working as intended, and it was! Armed with the location information, I whipped up an image loading convenience function (which actually paid attention to the errors instead of tossing them in the _ bin.

func loadImage(file string) image.Image {
  reader, err := os.Open(file)
  if err != nil {
    log.Fatalf("error loading %s: %s", file, err)
  }
  img, _, err := image.Decode(reader)
  if err != nil {
    log.Fatalf("error loading %s: %s", file, err)
  }
  return img
}

Image Manipulation

With that, my new loop looked something like this:

baseImage := loadImage(imageFile)
chrisFace := loadImage(chrisFaceFile)

bounds := baseImage.Bounds()

finder := facefinder.NewFinder(haarCascadeFilepath)
faces := finder.Detect(baseImage)

// Convert image.Image to a mutable image.ImageRGBA
canvas := image.NewRGBA(bounds)
draw.Draw(canvas, bounds, baseImage, bounds.Min, draw.Src)

for _, face := range faces {
  draw.Draw(
    canvas,
    face,
    chrisFace,
    bounds.Min,
    draw.Src,
  )
}

And what better to test it on, than a photo of the man himself!

Okay, real talk, that worked out way better than I expected for a first run attempt. Tangible progress!

First I looked at how to get rid of that black background. I was using a PNG with background transparency, so surely there was a way. A little Googling and I stumbled across draw.Over for the draw.Draw function. I swapped that in for the draw.Src I was using, and voila!

I could probably have feathered the cutout a little better, but a little voice in my head told me that shittier is the way to go here.

Okay! Next I just needed to scale down the face a bit. I could tell that if I fit the face into the rectangle it was expecting, it wouldn’t line up properly. Being a face detector, not a head detector, the rectangles I was getting back weren’t actually suitable for full head replacement. I whipped up a quick function to increase the image.Rectangle by a given margin, plugged in some numbers to dial it in a bit, and landed on 30%.

Once that was out of the way, I moved on to image resizing/fitting. I looked around a bit and found several options, but I landed on disintegration/imaging because it had a simple imaging.Fit function and offered some other transform operations like horizontal mirroring. I didn’t have a whole lot of source images for faces, so I figured random mirroring would give me twice as many options for the output.

After importing, my new loop looked something like this:

for _, face := range faces {
  // Pad the rectangle by 30 percent
  rect := rectMargin(30.0, face)

  // Grab a random face (also 50/50 chance it's mirrored)
  newFace := chrisFaces.Random()

  chrisFace := imaging.Fit(newFace, rect.Dx(), rect.Dy(), imaging.Lanczos)

  draw.Draw(
    canvas,
    rect,
    chrisFace,
    bounds.Min,
    draw.Over,
  )
}

I gritted my teeth and ran it against some new test images, and huge success!

At this point, I knew I was onto something good.

Slack Integration

I turned the face manipulation code into a runnable binary, which I intended to wrap with a Slack bot. I had already developed it as a binary for easy testing, and I didn’t want to spend the time converting it to an importable package for the Slack bot. At this point, I considered the face replacer “good enough” and moved on to focus on building the Slack bot to run it.

Once again, I turned to Google.

The very first result had everything I needed, with minor modifications for downloading and uploading files. I spent a lot of time reading Slack’s API documentation and a lot more time cursing at it, throwing things at the proverbial wall until something stuck, and finally I got to this:

NICE

The first iteration used Slack uploads, but being on a free Slack tier meant this was not ideal. I changed that to save the output locally onto my server, and then link that in Slack. Since Slack will auto-expand most image links, this wound up being an equivalent user experience for most people without as much risk of me getting in trouble with the office bigwigs.

With easy access to the process, I was able to more easily experiment with the face replacer. I realized that if it didn’t find any faces at all, it would just echo the same image back. That’s no fun! So I threw this in after the loop:

if len(faces) == 0 {
  // Grab a specific face and resize it to 1/3 the width
  // of the base image
  face := imaging.Resize(
    chrisFaces[0],
    bounds.Dx()/3,
    0,
    imaging.Lanczos,
  )

  face_bounds := face.Bounds()

  draw.Draw(
    canvas,
    bounds,
    face,
    // I'll be honest, I was a couple beers in when I came up with this and I
    // have no idea how it works exactly, but it puts the face at the bottom of
    // the image, centered horizontally with the lower half of the face cut off
    bounds.Min.Add(image.Pt(
      -bounds.Max/2+face_bounds.Max.X/2,
      -bounds.Max.Y+int(float64(face_bounds.Max.Y)/1.9),
    )),
    draw.Over,
  )
}

Which gave me this:

A pretty good solution, if I do say so myself.

Alright, that’s all the pieces put together, but what would my co-workers think of it? I had just gone from concept to prototype in a single evening, and none of them had any idea of what I had in store for them.

Introducing @Chrisbot

My manager up until now had been the most prolific manual Chris-shopper.

Sorry Mat, automation eats all jobs eventually.

The man himself was quite pleased, though.

And it wasn’t long before the whole office was posting photos to @Chrisbot.

I was pleasantly surprised at how it seemed to correctly handle face overlaps, drawing furthest faces first. This is purely a coincidental side effect of the ordering of the rectangles returned by the go-opencv library, but one I’m glad for.

But while the automated faceshopping has greatly increased the quantity of Chris in our Slack, there are some who maintain that the personal touch will always prevail.

I have to concede, in some cases that still holds true.