A Machine for Living In
One Day (24 hours in 24 minute single channel video)
This projects uses newly available computational and sensing tools to study the home as a site of intimate life. The title invokes Le Corbusier’s modernist framing of the house as a machine to interpret the promise of contemporary smart home technologies. The project has two distinct phases: the construction and inhabitation of a functional smart home system, followed by an exhibition of processed data as a multi-part digital art installation. In a process of joint human-machine authorship, this system produces a complex portrait of the home: as a space of language, intimacy, bodily practice, and quotidian narrative. Compositionally, it contrasts utopian illusions of beautiful, frictionless utility with artistic strategies generating insight into the messy, material realities of the everyday.
My written dissertation on the project, “Machines for Living,” is online here.
Documentation
One Day (excerpt from machine analyzed audio and video recorded in the house)
Sensor Narrative (video excerpt)
Installation
Installation View
What is it about today’s homes? (model smart home)
Thermal Blanket
Motion Trails
Thursday Evening, Family Dinner
Installation View
Installation (left: 3d scan, right: one day, bottom: floor plan)
House Scans (5 min single channel video)
Floor Plan
Installation View (right side)
Sensor Narrative and Sensing Sites
This project was exhibited in the Foster Art Gallery at Westminster College in 2017.
Sensing and Analysis
Sensing Points
Audionode with MEMS microphone and VAD software
Spectograph of human speech
Voice Activity Detection
Installed Sensor
Sensor Logs
A Machine for Living In
This project uses embedded, situated computing to study the home as a site of intimate life. After constructing and inhabiting a multi-modal smart home system, resulting data drives a generative art installation. The result is a complex, machine-mediated portrait of the everyday. Here, two videos—overhead computer vision annotating occupancy behavior, and generative video imagining new events in that space—explore the role of in-home AI and the new perspectives provided by machine intelligence.
Thursday Evening, Family Dinner
Motion Trail Drawings
My written dissertation on the project, “Machines for Living,” is online here.
Data Science
A Machine for Living In
Sensing Strategy:
- Physical Sensors
- vibration, temperature, humidity, beam break, physical switch, Passive Infra Red (PIR)
- Audio Nodes
- ICS43432 MEMS microphone interface via I2S
- recording 24bit 48kHz audio
- running on-device voice activity detection written in python (webrtcvad)
- Video nodes
- chip camera
- on-device motion detection using motion field thresholding from h264 encoder block in the SOC.
- v2: on-device image analysis with movidius VPU
- FLIR camera
Analysis:
- Audio Analysis:
- feature extraction (12 points)
- 4 Mel-Frequency Cepstral Coefficients
- Spectral Centroid
- 6-point Chromagram
- Fast Fourier Transform crest
- clustering
- k-Means
- HDBSCAN
- feature extraction (12 points)
- Video Analysis:
- background removal and motion path estimation (knn, MOG, HOG with opencv)
- v2: unwarping and Google Image Net classifier
- Sensor Analysis:
- time distribution of sensor activations over the day