It seems people expect projectors to magically improve illumination - they're not meant for that. Projectors are meant to produce sharp cut offs, to reduce glare and allow light only inside the desired shape/profile. Projectors also have reflectors with gather and focus the light, which is then defined by a shade/shield which defines the shape of the beam. Our office projectors are a good example - a rectangular frame of light which would otherwise be a broader spotlight. Good lighting is most dependent on a well designed reflector. Remember the Pulsar 220 - it has a projector for low beam, but the high beam is not and the high beam is one of the few beams on motorcycles that rivals cars. In fact, I think projectors waste/block a lot of light produced by the bulb ; the Pulsar 220 low beam is only a little better than a Karizma (old model with single headlight) low beam - I know because I've used both for 5 and 10 years.
The differences type of luminescent in terms of physics principle (black body radiation for halogen/incadescent vs electroluminscence for HID and LED ) are what cause the colour temperature to differ among them.
Incandescent bulbs seem inefficient to you - that's because we can't see in the infrared spectrum. Bulbs emit light in continuous frequencies, from infra-red to UV , most of the energy, ~90% is in fact concentrated in the infra-red band emission - which we can't see. If we could see in IR, all of us would unanimously prefer incandescent bulbs over LED and HID.
Incandescent bulbs emit across the spectrum, but less lumens for each wavelength/frequency band. For example, of 2000 lumens in the visible range, 300 lumens red, 300 lumens orange, 400 lumens yellow, 350 lumens green , 400 lumens blue , 250 lumens violent - all add up to 2000 lumens. Our eyes can see best in yellow-green and red-yellow range, where we have the most cones.
HID and LEDs have more selective spectral emissions. The color temperature of HID depends on the gas used, which upon excitement by electrical stimulation, responds by emitting packets or quanta of light in certain wavelengths , a sodium vapour emits characteristic orange glow, while mercury emits its chracteristic bluish white light - that's its chemistry and it can only emit in that narrow frequency band.
Likewise, LEDs too have selective wavelength - the electrical energy that stimulates the electrons of photoluminescent materials exits only in certain quanta, which depending on type emits a different color/wavelength, which is even narrower a band than HID. Now, the regular world is made up of varied materials, which absord most of the incident light and reflect back only small part of it (which is essentially why we see colors), this also means most of the light we throw is lost, absorbed and only a small fraction is useful as it returns to the eyes. Now let's say material A absorbs all light but reflects yellow, material B absorbs all light except reflecting blue, material C absorbs all light except returning red, material D absorbs all light except reflecting back green. So you're throwing spectrum materials A, C and D do not reflect.
Now with halogens, you're bound to get some light reflect back from some of various objects upon which the light is incident - less light of each wavelength , but you'll get a definite strong reflection from some part of spectrum and the halogen/incandescent throws a lot of wavelengths at everything. So even though most light is absorbed, some is reflected back helping you see. With HID/LED, that light thrown in of a narrow band of frequencies, with the bulk of it in the blue end. So objects like material A , C and D have little of their reflective frequencies to send back to you. While signboards and such reflective luminescent are designed to reflect light in the wavelengths we see (halogens also light these up well, but they're especially highlighted in LED/HID light), the rest of the world is not so conducive.
Go back to that sodium vapour street light -it's very bright when you look at it directly, but it shows little of the world. That's because it's emitting such a narrow band of wavelengths, most of it is not reflected but absorbed by whatever it's incident. You can't tell colors well in this light, that is what is meant by CRI - color rendering index, which is great rather best for natural light which has the widest spectrum.
Add to that, higher frequency radiation scatters more , especially in the presence of particulate matter - such as moisture, smoke, dust, making LED and HID more ineffective.
We're fooled by the immediate bright spot in front of us, in close proximity to the vehicle in demos, but what we need is less bright immediate proximity and more light focused at a distance, with say 90% of the light spread from 25-100 meters ahead, only 10% from 0-25 meters in front of use.
Too bright light focus in the immediate foreground also forces our pupils to contract, further reducing light entering the retina.
About why even the otherwise superbright LEDs seem inadequate in the face of oncoming traffic (with high beams), I've written about this earlier here :
http://www.team-bhp.com/forum/motorb...ml#post4160131
As for why wet conditions reduce the visibility, it's to do with increased internal/random reflection cause by the water film on wet surfaces. Ever see more strong reflections in water puddles or even on wet roads caused by lights of other vehicles , that you don't see as strong when the road is dry? That's because even less of the light from his headlights is reflecting back at him. Read more about this here :
https://www.victoria.ac.nz/scps/abou...kerwhenwet.pdf