robotics//sensor//camera
A camera is a sensor that measures the light intensity reaching each pixel of an array of photodiodes, and in robots and machines it is the cheapest source of rich information: texture, colour, text and signs, at 30 to 60 frames per second. Physically it is a chip with a resin window over the photodiodes. The ADC and the image signal processor (ASIC) sit directly below the sensor die, and an image travels from the light to the screen in five steps:
A camera is a sensor that measures the light intensity reaching each pixel of an array of photodiodes, and in robots and machines it is the cheapest source of rich information: texture, colour, text and signs, at 30 to 60 frames per second. Physically it is a chip with a resin window over the photodiodes. The ADC and the image signal processor (ASIC) sit directly below the sensor die, and an image travels from the light to the screen in five steps:
1Light hits the photodiodes2The ASIC ADC converts voltage to bits1.5 GB/s3The SoC processes the imageASIC image processing4RAM holds the decoded colors5The screen shows them
Geometrically, the pinhole camera model says how a point of the world lands on a pixel. A point XXX seen from a camera with orientation RRR and position ttt projects to the pixel (u,v)(u,v)(u,v) through the internal parameters KKK (focal length, image centre):
λ[uv1]=K [ R∣t ][X1].\lambda \begin{bmatrix} u \\ v \\ 1 \end{bmatrix} = K\,[\,R \mid t\,]\begin{bmatrix} X \\ 1 \end{bmatrix}.λuv1=K[R∣t][X1].
The factor λ\lambdaλ is the depth, and the projection throws it away: a single camera measures the direction to each point and loses its distance. Depth comes back from a second camera (stereo vision), from motion combined with an IMU (visual-inertial odometry) or from a learned estimate.
It fails in low light, in glare, with motion blur and on plain, textureless surfaces. Many cheap sensors have a rolling shutter that exposes the rows one after another, so under fast motion vertical lines lean and a propeller looks bent, which corrupts any geometry computed from the image.
Its data rate is large: a 1280×720 grey image at 30 frames per second is about 28 MB/s raw, which is why vision runs on a companion computer and not on the flight controller (perception).
It needs three calibrations, all with photos of a known pattern or in motion: the intrinsics KKK and the lens distortion, the extrinsics (where it is mounted relative to the body and the IMU), and its time offset (sensor calibration, time synchronization).
An event camera reports only changes of brightness, pixel by pixel, with microsecond resolution and no frames; it handles fast motion and extreme lighting and is still mostly research.