I_AM_THE_FUCKING_INPUT (2023)

To care for my type 1 diabetes I use an automated insulin delivery (AID) system. This involves the use of a continuous glucose monitor (CGM) that communicates with an insulin pump. The insulin pump uses this blood sugar data (and some other info) to make automated decisions for dosing me with insulin.

These AID systems require converting my blood into data in order to predict future blood sugars, to be used as feedback to regulate my blood sugar. As this data is processed by different algorithms, its errors, omissions, and assumptions end up creating artifacts in the final results. These artifacts have real world consequences when taken out abstraction and applied to my body.



For I_AM_THE_FUCKING_INPUT, I recorded videos of myself doing many different motions. I selected and edited 12 clips to be the same length. For each clip, I passed the video through an algorithm (Google's Mediapipe) to create a mask to isolate just my body. The mask generated is imperfect, meaning the cutout ends up including parts of the background or clipping parts of my body off. I placed this masked video over a green background.



I then used Pix2PixHD to do Next-Frame Prediction^[1-2]. This involved training 12 models (one for each video). Once done, I fed a starting frame from the original video into the model, generating the next frame. Then, I took the generated frame and fed it back into the model again. I repeated this until I had generated enough frames to create a video of specific length.

The model has poor context, only having the previous image to generate the next from. For the videos I chose, the models predication fails quickly. The resulting videos move from my original video to abstraction, created from the models prediction.



I put all the new videos in a four by four grid. I used this video for two more processes.



First, I took the video into 3D render software. I first removed the green to create a transparent background. This way, I could use the video as a texture for a plane, and each could cast a shadow. I placed the video so each copy of me would be inside its own box inside a model I made.



Second, I processed the video to create an outline of each frame, saving it as a PNG. I wrote scripts to bring each PNG into 3D software, convert in to an SVG, give it thickness, and move it up in the Y axis equal to its new height. The thickness of each layer is equal to the thickness of a print layer in the Formlabs SLS printer I had access to. The final model was a vertical stack of every frame of the video.



I printed the model on the Formlabs SLS printer and recorded the printers camera output. I edited and sped up the final printing video to create stop-motion version of the four by video I had created.


Last, I had the 3D print from the SLS printer. The result was a physical object with artifacts resulting from the all the processing.

[1] JC Testud, Video Generation With pix2pix, https://medium.com/@jctestud/video-generation-with-pix2pix-aed5b1b69f57
[2] Derrick Schultz, Next Frame Prediction using Pix2PixHD, https://colab.research.google.com/github/dvschultz/ml-art-colabs/blob/master/Pix2PixHD_Next_Frame_Prediction.ipynb
Materials and Tools:
Touchdesigner, Blender, Mediapipe, pix2pix, Formlabs SLS printer