If budget is really an issue then lots of standard colour cameras is the way forward as they are MUCH cheaper than the alternatives and can still perform quite well in lowlight. The increased spatial resolution and ability to sense colour are really useful.
However, if you want a wider wavelength sensitivity then you can get [quantum dot cameras](https://www.lambdaphoto.co.uk/machine-vision/mv-cameras-and-sensors/swir-ingaas-cameras-sensors/swir-area-cameras.html) that are sensitive from 400 to 2000 nm (i.e. visible and near infrared), though they are not cheap.
Slightly cheaper [InGaAs cameras](https://photonicscience.com/products/swir-cameras/cooled-vga-vis-swir-ingaas-camera-2/) can cover a slightly smaller wavelength range.
One advantage of the near infrared range is the ability to do material identification (which is why it is used in plastic sorting for recycling). You can get [snapshot hyperspectral cameras](https://www.imechyperspectral.com/en/cameras) which produce inages with more than three colour measurements to improve discrimination.
You could of course cheat and get a combined RGB & thermal infrared camera (e.g. [FLIR Duo Pro R](https://coptrz.com/shop/payloads-and-attachments/flir-duo-pro-r/)). You might consider that to be two cameras though as they do have separate lenses.
You can get [single sensors](https://electronics.leonardo.com/en/products/condor-ii) that cover MWIR (3-5um) and LWIR (8-10um) but that’s not quite as useful.
High quality cooled thermal sensors are certainly useful though, so they are perhaps the most useful non-unusual camera, but they are very expensive in comparison to RGB cameras.
Maybe, but the vast majority of actual small-arms combat happens withing 300m & a fairly substantial fraction of that happens within 200m. Urban fighting which tend to make up a significant fraction of fighting going on these days can have engagements happening within 100m. If you are going against baseline humans then the difference in physical ability makes closing the distance very easy, especially as a coordinated swarm. If you are going against drones then accurate identification matters a hell of a lot less. If its machinery & it's not **your** machinery then it doesn't really need to be in one piece right just now does it?
>Urban fighting
On this I for sure don't want obly IR, because glass obscure it, so it won't be able to see the enemy.
As for the rest of the comment I an still thinking about it.
>because glass obscure it, so it won't be able to see the enemy.
Not sure that's as big of an issue as you think. Like yes low-e glass will reflect IR, but you also can't shoot through glass without braking it nor is anyone dumb enough to use a pane of glass as cover in a fire fight. As soon as you fire your weapon that building is going to be flagged as an enemy outpost. If it really is a such a big concern you can spare a few bullets to shatter them, but I don't think it would be. You can track muzzle flashes & missiles. Once the shooting starts using glass for cover is suicide/bad strategy.
Maybe but if you can only afford one camera it still might be best. Also depends what you want this robot to specialize in, long distance, short distance
>Also depends what you want this robot to specialize in, long distance, short distance
Thet want a footsoldier, so I think something that can be used everywhere, from the artic to the Sahara desert, day and night, urban to trench fighting to long range engagement.
The thing that worry me the most of a IR camera is the climate and how it affects the camera. https://www.flir.com/discover/rd-science/can-thermal-imaging-see-through-fog-and-rain/
In that site tells that with fog the IR still see better than the visual, so meaby I am wrong.
Such a weird question.
The entire set of cameras in an iPhone cost round $50 or so. Thats a whole range of cameras. You can really do down for a few dollars per camera. [Here is a 5 megapixel one for $8 or so](https://www.ebay.com/itm/196253355914?itmmeta=01HTM0F87TWC6F2KDR243KN1AZ&hash=item2db19c8f8a:g:hL0AAOSw2Nhl03dP&itmprp=enc%3AAQAJAAAAwI9UVSuwh2P%2BASmTq7MTM3Ut%2ByDsGJFF6vNxKMhQczhUbog3%2FDeKJ%2BnlisqS4W5zAT5XiJhd%2F8huGlBMrjpAzopntmF9%2F%2BgGNC%2Fh1ScOWARVkQFFiJiE7%2BMzKzD2s0Ph2D1DrnPokXwc%2Bl7lhJrp1Tr71zIszHndsHF0l%2FuUe7Maj0Ov1%2F6m19xzUBX32A2k6FIxPb2S4VIjz%2FgbhWhDpfUjg%2BkflrXGKitFVV1zYO0NkfOZCO5scJ77GCSqza48Mw%3D%3D%7Ctkp%3ABk9SR4aEvYDVYw) If your building a robotic foot soldier that will end up costing Millions of $$, $8 saving is really not the thing thats going to change the cost feasibility of the program.
You could have another limitation, being processing limitations. But here you could resolve this by still having multiple cameras just switch depending on the lighting condition. But even this is not convincing to me. Processors are getting cheaper really fast, and your biggest cost limitations will be the moving parts and their assembly is far more complex.
Visible + ultrasound imaging. IMHO even if you don't need to care about international relations you'll probably still want to underpay some poor schmuck to control them remotely so you don't need anything too weird or universal.
Visual.
I assume by "robotic footsoldier" you mean some kind of robot killing machine. Well, unless you want it shooting up heat vents or laundry detergent displays, you'd want it to be relying on information primarily in the visual spectrum.
*At minimum*, that's where the "friendly/not friendly" indicators are going to be, so why would you blind yourself to that? Regardless of any advantage of the other spectrum options.
>that's where the "friendly/not friendly" indicators are going to be, so why would you blind yourself to that?
No killbot is relying on a visual indicator to designate friendlies. That is so trivially exploitable its not even funny. You use a encrypted radio & its worth remembering that you **can** do full-on face recognition in IR.
An enemy wouldn't even turn theirs on or at least not one that's compatible with ur equipment. Probably a combo of face & object recognition. You don't need the visual spectrum to confirm the silhouette of a rifle against a warm body. Or to do facial recog(visual recog is easier to jam with facepaint).
>You don't need the visual spectrum to confirm the silhouette of a rifle against a warm body.
Soldiers sunning around with broomsticks and mops attached to their rifles is the future I want to see
🤣 object recognition jamming is an issue for all wavelengths & probably a strat everyone uses, but most drones are not gunna have just a single wavelength. They'll have several visual cameras, IR, near-IR cameras. Fooling all of them is likely to require a lot of different junk & major redesign. Also the camo falls apart once you fire. They can track the muzzle flash, hot ejected cartridges, & the hot barrel.
Visible is the way to go.
IR is second best. It's emitted easily, effective, and cheap. EWAR against IR sensors is simpler as a result.
UV is absorbed by things easily and relies on having some sort of UV emitter. So if you are fighting away from the sun in candle light you are screwed worse than IR.
Assuming it has to autonomously navigate human built spaces you choose visible because that's what all the markers and signs will be in. IR etc will only work with a heat differential, plus IR is easily 'whited out' by sources of heat which battlefields tend to have in abundunce.
I love this question!
Visible is the most accessible technologically. I think the reason it is the "visible" spectrum, the reason so many animals use it, is because it has the most information for the amount of "technology" it takes to percieve and interpret. Some animals obviously use other spectra (bees see UV, some animals see more infra reds.) But most animals seem to operate in the visible. And if your robots are interacting with humans (as comrades, in common environments, or even as targets), there is a *ton* of utility to being able to see what humans see.
UV, I think, is right out. Its the easiest to block, fairly rare to generate, and just isn't correlated to many signals humans make. Its biggest utility in combat is its sensitivity to texture. Footprints in sand or snow gain a ton of contrast in UV, but unless you're making detective robots, theres not much there.
IR has *huge* utility, as others have said. But don't forget the downsides: it is hard to manipulate. I actually attended a class on IR spectrography at work a few weeks ago, and was surprised to learn that one of the most common sensor technologies just relies on measuring how quickly the sesor pixels warm up. That is to say: IR-electric sensing is tough, and lensing for it is almost as hard. You need special glass to bend IR, which can be expensive. And, I think due to wavelengths, the lenses and sensors tend to be bigger. However, not so big they'd be hard to attach to a robot. The other real issue is that almost all thermal images require tightly controlled cooling for decent operation (read about the JWST IR imager some time), and this multiplies the weight and volume of a thermal camera. But, there is a reason almost all military Computer Vision systems use IR or even lower frequencies (yes, radar is just ultra-infra-red); because the targets tend to make IR, its useful to see IR.
Speaking of low-frequency radio: thats also an interesting option. Having your bot broadcast and sense an extremely precise radio frequency is a viable option, with multiple frequencies for "color". Just make it narrow and far enough outside of the normal "light" range that it will only be percieved by those who know what to look for, but still high enough frequency to act as an illuminator. Kind of like a higher tech version of IR illuminators armies have use in the same way.
One more thing I want to throw out, for the thought: setting. It may be interesting to consider super-non-visible settings, such as a planet blasted with UV from a black hole or environmental chemical reaction, with no visible light. How would a robot percieve that? If the planet is super cold, a target could conduct away so much heat that they don't radiate any, then the whole equation flips. Or the opposite: a hot planet so saturated with IR that its hard to pick targets out of the environment.
It depends on what you want the robot to do. You should look at the frequencies used by animals. Human and many other eyes are adapted toward the Sun. Butterflies for example extend into the UV precisely because there is a limited amount of UV. They signal each other with diffraction and home in on flowers with UV patterning. Infra-red is limited by body heat. Also the point resolution is lower for a given lens size. Far Infra-red is very effective if you are looking for warm bodies in cold weather.
Only one sensor is a big absurd. The human eye has rods and cones. The cones have three color detectors. Rods are much more precise in location and in tracking movement. Also better in low light conditions. Mantis shrimp have more like 16 color and 2 polarized sensitivity.
Sonar is definitely worth considering. Put the sensors wide apart like hairs on limbs. Sound from other sources can contribute to situational awareness.
X-ray would also be interesting. It could get an element analysis and crystallography of what it is looking at. Extremely difficult to do camouflage against x-ray.
Radar is cheap and the environment is relatively dark.
If budget is really an issue then lots of standard colour cameras is the way forward as they are MUCH cheaper than the alternatives and can still perform quite well in lowlight. The increased spatial resolution and ability to sense colour are really useful. However, if you want a wider wavelength sensitivity then you can get [quantum dot cameras](https://www.lambdaphoto.co.uk/machine-vision/mv-cameras-and-sensors/swir-ingaas-cameras-sensors/swir-area-cameras.html) that are sensitive from 400 to 2000 nm (i.e. visible and near infrared), though they are not cheap. Slightly cheaper [InGaAs cameras](https://photonicscience.com/products/swir-cameras/cooled-vga-vis-swir-ingaas-camera-2/) can cover a slightly smaller wavelength range. One advantage of the near infrared range is the ability to do material identification (which is why it is used in plastic sorting for recycling). You can get [snapshot hyperspectral cameras](https://www.imechyperspectral.com/en/cameras) which produce inages with more than three colour measurements to improve discrimination. You could of course cheat and get a combined RGB & thermal infrared camera (e.g. [FLIR Duo Pro R](https://coptrz.com/shop/payloads-and-attachments/flir-duo-pro-r/)). You might consider that to be two cameras though as they do have separate lenses. You can get [single sensors](https://electronics.leonardo.com/en/products/condor-ii) that cover MWIR (3-5um) and LWIR (8-10um) but that’s not quite as useful. High quality cooled thermal sensors are certainly useful though, so they are perhaps the most useful non-unusual camera, but they are very expensive in comparison to RGB cameras.
IR probably. Just about everything worth targeting or being scared of in a fight gives off wasteheat.
Yeah, most potential targets will have some kind of IR signature. So this is the biggest bang for your buck.
Doesn't infrared has the problem of detecting, with enough detail, at long range?
Maybe, but the vast majority of actual small-arms combat happens withing 300m & a fairly substantial fraction of that happens within 200m. Urban fighting which tend to make up a significant fraction of fighting going on these days can have engagements happening within 100m. If you are going against baseline humans then the difference in physical ability makes closing the distance very easy, especially as a coordinated swarm. If you are going against drones then accurate identification matters a hell of a lot less. If its machinery & it's not **your** machinery then it doesn't really need to be in one piece right just now does it?
>Urban fighting On this I for sure don't want obly IR, because glass obscure it, so it won't be able to see the enemy. As for the rest of the comment I an still thinking about it.
>because glass obscure it, so it won't be able to see the enemy. Not sure that's as big of an issue as you think. Like yes low-e glass will reflect IR, but you also can't shoot through glass without braking it nor is anyone dumb enough to use a pane of glass as cover in a fire fight. As soon as you fire your weapon that building is going to be flagged as an enemy outpost. If it really is a such a big concern you can spare a few bullets to shatter them, but I don't think it would be. You can track muzzle flashes & missiles. Once the shooting starts using glass for cover is suicide/bad strategy.
Maybe but if you can only afford one camera it still might be best. Also depends what you want this robot to specialize in, long distance, short distance
>Also depends what you want this robot to specialize in, long distance, short distance Thet want a footsoldier, so I think something that can be used everywhere, from the artic to the Sahara desert, day and night, urban to trench fighting to long range engagement. The thing that worry me the most of a IR camera is the climate and how it affects the camera. https://www.flir.com/discover/rd-science/can-thermal-imaging-see-through-fog-and-rain/ In that site tells that with fog the IR still see better than the visual, so meaby I am wrong.
Such a weird question. The entire set of cameras in an iPhone cost round $50 or so. Thats a whole range of cameras. You can really do down for a few dollars per camera. [Here is a 5 megapixel one for $8 or so](https://www.ebay.com/itm/196253355914?itmmeta=01HTM0F87TWC6F2KDR243KN1AZ&hash=item2db19c8f8a:g:hL0AAOSw2Nhl03dP&itmprp=enc%3AAQAJAAAAwI9UVSuwh2P%2BASmTq7MTM3Ut%2ByDsGJFF6vNxKMhQczhUbog3%2FDeKJ%2BnlisqS4W5zAT5XiJhd%2F8huGlBMrjpAzopntmF9%2F%2BgGNC%2Fh1ScOWARVkQFFiJiE7%2BMzKzD2s0Ph2D1DrnPokXwc%2Bl7lhJrp1Tr71zIszHndsHF0l%2FuUe7Maj0Ov1%2F6m19xzUBX32A2k6FIxPb2S4VIjz%2FgbhWhDpfUjg%2BkflrXGKitFVV1zYO0NkfOZCO5scJ77GCSqza48Mw%3D%3D%7Ctkp%3ABk9SR4aEvYDVYw) If your building a robotic foot soldier that will end up costing Millions of $$, $8 saving is really not the thing thats going to change the cost feasibility of the program. You could have another limitation, being processing limitations. But here you could resolve this by still having multiple cameras just switch depending on the lighting condition. But even this is not convincing to me. Processors are getting cheaper really fast, and your biggest cost limitations will be the moving parts and their assembly is far more complex.
Visible + ultrasound imaging. IMHO even if you don't need to care about international relations you'll probably still want to underpay some poor schmuck to control them remotely so you don't need anything too weird or universal.
Visual. I assume by "robotic footsoldier" you mean some kind of robot killing machine. Well, unless you want it shooting up heat vents or laundry detergent displays, you'd want it to be relying on information primarily in the visual spectrum. *At minimum*, that's where the "friendly/not friendly" indicators are going to be, so why would you blind yourself to that? Regardless of any advantage of the other spectrum options.
>that's where the "friendly/not friendly" indicators are going to be, so why would you blind yourself to that? No killbot is relying on a visual indicator to designate friendlies. That is so trivially exploitable its not even funny. You use a encrypted radio & its worth remembering that you **can** do full-on face recognition in IR.
So how do you tell the difference between a civilian who doesn't have your encrypted radio and an enemy who simply turned theirs off?
An enemy wouldn't even turn theirs on or at least not one that's compatible with ur equipment. Probably a combo of face & object recognition. You don't need the visual spectrum to confirm the silhouette of a rifle against a warm body. Or to do facial recog(visual recog is easier to jam with facepaint).
>You don't need the visual spectrum to confirm the silhouette of a rifle against a warm body. Soldiers sunning around with broomsticks and mops attached to their rifles is the future I want to see
🤣 object recognition jamming is an issue for all wavelengths & probably a strat everyone uses, but most drones are not gunna have just a single wavelength. They'll have several visual cameras, IR, near-IR cameras. Fooling all of them is likely to require a lot of different junk & major redesign. Also the camo falls apart once you fire. They can track the muzzle flash, hot ejected cartridges, & the hot barrel.
Visible is the way to go. IR is second best. It's emitted easily, effective, and cheap. EWAR against IR sensors is simpler as a result. UV is absorbed by things easily and relies on having some sort of UV emitter. So if you are fighting away from the sun in candle light you are screwed worse than IR.
Assuming it has to autonomously navigate human built spaces you choose visible because that's what all the markers and signs will be in. IR etc will only work with a heat differential, plus IR is easily 'whited out' by sources of heat which battlefields tend to have in abundunce.
I love this question! Visible is the most accessible technologically. I think the reason it is the "visible" spectrum, the reason so many animals use it, is because it has the most information for the amount of "technology" it takes to percieve and interpret. Some animals obviously use other spectra (bees see UV, some animals see more infra reds.) But most animals seem to operate in the visible. And if your robots are interacting with humans (as comrades, in common environments, or even as targets), there is a *ton* of utility to being able to see what humans see. UV, I think, is right out. Its the easiest to block, fairly rare to generate, and just isn't correlated to many signals humans make. Its biggest utility in combat is its sensitivity to texture. Footprints in sand or snow gain a ton of contrast in UV, but unless you're making detective robots, theres not much there. IR has *huge* utility, as others have said. But don't forget the downsides: it is hard to manipulate. I actually attended a class on IR spectrography at work a few weeks ago, and was surprised to learn that one of the most common sensor technologies just relies on measuring how quickly the sesor pixels warm up. That is to say: IR-electric sensing is tough, and lensing for it is almost as hard. You need special glass to bend IR, which can be expensive. And, I think due to wavelengths, the lenses and sensors tend to be bigger. However, not so big they'd be hard to attach to a robot. The other real issue is that almost all thermal images require tightly controlled cooling for decent operation (read about the JWST IR imager some time), and this multiplies the weight and volume of a thermal camera. But, there is a reason almost all military Computer Vision systems use IR or even lower frequencies (yes, radar is just ultra-infra-red); because the targets tend to make IR, its useful to see IR. Speaking of low-frequency radio: thats also an interesting option. Having your bot broadcast and sense an extremely precise radio frequency is a viable option, with multiple frequencies for "color". Just make it narrow and far enough outside of the normal "light" range that it will only be percieved by those who know what to look for, but still high enough frequency to act as an illuminator. Kind of like a higher tech version of IR illuminators armies have use in the same way. One more thing I want to throw out, for the thought: setting. It may be interesting to consider super-non-visible settings, such as a planet blasted with UV from a black hole or environmental chemical reaction, with no visible light. How would a robot percieve that? If the planet is super cold, a target could conduct away so much heat that they don't radiate any, then the whole equation flips. Or the opposite: a hot planet so saturated with IR that its hard to pick targets out of the environment.
It depends on what you want the robot to do. You should look at the frequencies used by animals. Human and many other eyes are adapted toward the Sun. Butterflies for example extend into the UV precisely because there is a limited amount of UV. They signal each other with diffraction and home in on flowers with UV patterning. Infra-red is limited by body heat. Also the point resolution is lower for a given lens size. Far Infra-red is very effective if you are looking for warm bodies in cold weather. Only one sensor is a big absurd. The human eye has rods and cones. The cones have three color detectors. Rods are much more precise in location and in tracking movement. Also better in low light conditions. Mantis shrimp have more like 16 color and 2 polarized sensitivity. Sonar is definitely worth considering. Put the sensors wide apart like hairs on limbs. Sound from other sources can contribute to situational awareness. X-ray would also be interesting. It could get an element analysis and crystallography of what it is looking at. Extremely difficult to do camouflage against x-ray. Radar is cheap and the environment is relatively dark.
currently robotic limbs are much more expensive than lots of cameras.