Go Back   Team-BHP > Under the Hood > Technical Stuff


Reply
 
Thread Tools Search this Thread
Old 20th February 2014, 11:41   #1
Newbie
 
Join Date: Feb 2014
Location: Bangalore/Goa
Posts: 20
Thanked: 32 Times
Exclamation Advice on designing a 'Sleep Detection System'

Hi all,
I am new to team-bhp. I have been a silent reader since 2009. I am currently pursuing my B.E. in electronics and communications and have decided to design a sleepy eye detection system as my final year project.

We( our project group) have approached a guy who has offered to guide us through. We have chosen MATLAB for the purpose of image processing. So far we have extracted the driver's face from the image and blackened out the background. We plan on triggering an alarm when the blink frequency of the driver's eyes fall below a certain level. I do not have much knowledge on image processing as I've never studied the subject before. Would really like to know the opinions of fellow members'.

Below is the original image and the image with the face extracted:
Attached Thumbnails
Advice on designing a 'Sleep Detection System'-original.png  

Advice on designing a 'Sleep Detection System'-extract.png  

nigel92 is offline   (7) Thanks Reply With Quote
Old 20th February 2014, 12:12   #2
BHPian
 
Kaumad's Avatar
 
Join Date: Jul 2010
Location: Chennai
Posts: 134
Thanked: 150 Times
Default re: Advice on designing a 'Sleep Detection System'

Good initiative Nigel.

I am not an expert in electronics but this..
Quote:
Originally Posted by nigel92 View Post
We plan on triggering an alarm when the blink frequency of the driver's eyes fall below a certain level.
might not be the best method to evaluate.

Many OEMs have been successful in developing Driver Alert Systems.
To being with, please google for "PERCLOS" - it is a "Psychophysiological Measure of Alertness As Assessed by Psychomotor Vigilance". There are many articles related to this. Maybe you can read up on them and post your ideas.

All the very best!!

Last edited by Kaumad : 20th February 2014 at 12:12. Reason: Removing tags.
Kaumad is offline   (4) Thanks Reply With Quote
Old 20th February 2014, 12:42   #3
Senior - BHPian
 
NetfreakBombay's Avatar
 
Join Date: Jan 2008
Location: Bombay
Posts: 1,385
Thanked: 336 Times
Default re: Advice on designing a 'Sleep Detection System'

This is an interesting project. Approach that you have taken seems feasible for a BE project.

Implementation will be based on "Feature Extraction" and analysis of those features.

To implement this, you need to:

1. Extract all "features" from image. Sample image is later in post.
2. Identify Eyes among the features. (Initially you can train the system for specific users).
3. Keep doing that in a loop for every "frame" in video.
4. If eyes change from open to close between frame, tag that as blink.

To do that, you can use few open source libraries. Examples are:


http://www.chrisevansdev.com/compute...-opensurf.html (Read "Notes on the OpenSURF Library" document)
Open CV : http://opencv.willowgarage.com/wiki/


Example image:

Name:  Untitled.png
Views: 5257
Size:  693.2 KB
NetfreakBombay is offline   (5) Thanks Reply With Quote
Old 20th February 2014, 13:56   #4
BHPian
 
muditgrover's Avatar
 
Join Date: Oct 2010
Location: Greater Noida
Posts: 193
Thanked: 131 Times
Default re: Advice on designing a 'Sleep Detection System'

You can use a Raspberry Pi and use its RaspiCam module which can be mounted on the windscreen. You can use OpenCV to detect when the drivers eyes are on the road and have a beep when they are not..

I have bought a CubieTruck ARM board for a project, let me know if you need any help.
muditgrover is offline   (2) Thanks Reply With Quote
Old 20th February 2014, 14:08   #5
Senior - BHPian
 
DerAlte's Avatar
 
Join Date: Mar 2007
Location: Bangalore
Posts: 8,076
Thanked: 2,876 Times
Default re: Advice on designing a 'Sleep Detection System'

Good project!

Feature Extraction, as @netfreakBombay has described it, is the way to go. But, it is neither the only step, nor the starting step. Matlab is not the right environment, or rather - Matlab is only the programming environment. You have to have an independent executable for dynamic analysis, and you have to write a lot of code. Don't assume someone else has already written the code which you can conveniently use as modules or library calls. Anyhow, this being a college project, coding is not the objective (availability of time in a semester program will be an issue for you, apart from resources - TANSTAFL) nor the greatest achievement - as long as you prove your algorithm is sound.

The starting point is the physical algorithm, as @kaumad has pointed out. There is plenty of information available on how sleepiness is detected - some of which only humans can figure out, and some which can be distilled to a mathematical algorithm. To give you a clue, what you are looking for is not blinking 'rate'. Some people blink not so often, yet others do it rapidly - something to do with dryness of the cornea. You should look for blinking 'behaviour': *how* the eyelids open/close - sometimes fast, sometimes slowly, and sometimes not at all (open/shut). Your first endeavor has to be detection of eyelid, after detecting the eye region (that is relatively simple).
DerAlte is offline   (3) Thanks Reply With Quote
Old 20th February 2014, 15:14   #6
Senior - BHPian
 
NetfreakBombay's Avatar
 
Join Date: Jan 2008
Location: Bombay
Posts: 1,385
Thanked: 336 Times
Default re: Advice on designing a 'Sleep Detection System'

nigel92,

Also look at this paper. It describes ways to extract Eye contours from images. A solution can be built around it (If your prof / guide agrees with approach).

http://citeseerx.ist.psu.edu/viewdoc...=rep1&type=pdf
NetfreakBombay is offline   (3) Thanks Reply With Quote
Old 20th February 2014, 15:14   #7
Newbie
 
Join Date: Feb 2014
Location: Bangalore/Goa
Posts: 20
Thanked: 32 Times
Smile

Mod Note : Please use the EDIT or MULTI-QUOTE buttons instead of typing one post after another on the SAME THREAD!.

To know how to multi-quote, click here (How to MULTI-QUOTE (when replying to a thread) on Team-BHP).

Quote:
Originally Posted by muditgrover View Post
You can use a Raspberry Pi and use its RaspiCam module which can be mounted on the windscreen. You can use OpenCV to detect when the drivers eyes are on the road and have a beep when they are not..

I have bought a CubieTruck ARM board for a project, let me know if you need any help.
As of now we have planned to use a FPGA kit to dump the code and trigger the alarm.

@netferakbombay,Our guide is making it look relatively simple. I have no prior knowledge of image processing. I hope he's guiding us in the right direction. Our progress so far as seen from the first post is extracting the face. Now, our guide's approach is that the eye region is the only region in the image where the sequence of skin-white-black-white will exist. So he says that if this combination exists that means that the eye is open, if not, the eye is closed.

@Kaumad yeah i get your point. Maybe the speed of the blink will be better.

Appreciate the help

Nigel

Last edited by GTO : 21st February 2014 at 15:29.
nigel92 is offline   Reply With Quote
Old 20th February 2014, 15:46   #8
Senior - BHPian
 
scopriobharath's Avatar
 
Join Date: Nov 2007
Location: Chennai
Posts: 3,565
Thanked: 1,175 Times
Default re: Advice on designing a 'Sleep Detection System'

Here is a brief logic that i would use - by the by i used to design robots in my college days
  • Capture face from a camera.
  • Process two parts of the image namely eye lid movement (use amount of white color of eye as a baseline for identifying whether eyelids are closed or open).
  • Secondly, capture sudden facial movement, similar to that of a person falling asleep and waking up.
  • Process these images and make a microprocessor send out a beep / sound an alarm.
Of course, a nutty professor at college might ask you, how would this system work if one is driving with sun glasses.

The ones Mercedes / volvo use are through high end electronic systems, that detect, suddent throttle changes, lane movement (without indicators), steering veer and a host of other things like car approaching median in a turn etc and apply brakes. There is a sensor at every logical junture like steering rod, brake, bumpers etc which send signals to a central "Computer/"Chip" and that chips does the secondary task of sounding an alarm or slaming the brake, or readying the airbag for deployment.
scopriobharath is offline   (2) Thanks Reply With Quote
Old 20th February 2014, 16:03   #9
Newbie
 
Join Date: Feb 2014
Location: Bangalore/Goa
Posts: 20
Thanked: 32 Times
Default re: Advice on designing a 'Sleep Detection System'

Quote:
Originally Posted by scopriobharath View Post
Here is a brief logic that i would use - by the by i used to design robots in my college days
  • Capture face from a camera.
  • Process two parts of the image namely eye lid movement (use amount of white color of eye as a baseline for identifying whether eyelids are closed or open).
  • Secondly, capture sudden facial movement, similar to that of a person falling asleep and waking up.
  • Process these images and make a microprocessor send out a beep / sound an alarm.
Of course, a nutty professor at college might ask you, how would this system work if one is driving with sun glasses.

The ones Mercedes / volvo use are through high end electronic systems, that detect, suddent throttle changes, lane movement (without indicators), steering veer and a host of other things like car approaching median in a turn etc and apply brakes. There is a sensor at every logical junture like steering rod, brake, bumpers etc which send signals to a central "Computer/"Chip" and that chips does the secondary task of sounding an alarm or slaming the brake, or readying the airbag for deployment.

@scorpiobharath, the part about the driver wearing sunglasses did cross our mind but we still went ahead with the idea. Compared to the sophisticated systems used in Mercs/Volvos this one will be much cheaper if successful. Our guide has assured us that he will complete the project by 31st March. Have my next class on Monday, till then have to try and detect the skin-white-black-white combination in the image.

Nigel
nigel92 is offline   (1) Thanks Reply With Quote
Old 20th February 2014, 16:21   #10
Senior - BHPian
 
scopriobharath's Avatar
 
Join Date: Nov 2007
Location: Chennai
Posts: 3,565
Thanked: 1,175 Times
Default re: Advice on designing a 'Sleep Detection System'

Quote:
Originally Posted by nigel92 View Post
@scorpiobharath, the part about the driver wearing sunglasses did cross our mind but we still went ahead with the idea. Compared to the sophisticated systems used in Mercs/Volvos this one will be much cheaper if successful. Our guide has assured us that he will complete the project by 31st March. Have my next class on Monday, till then have to try and detect the skin-white-black-white combination in the image.

Nigel
This logic should work - When the face is captured on turn of the key or slotting into gear, the wide open eyes would be captured. Due to fatiugue, they eyes close over the period of the drive. This picture could be used as a reference to capture fatigued eye. If the initial eye itself is blurry, it is probably because the person is under influence.
scopriobharath is offline   Reply With Quote
Old 20th February 2014, 17:11   #11
Senior - BHPian
 
NetfreakBombay's Avatar
 
Join Date: Jan 2008
Location: Bombay
Posts: 1,385
Thanked: 336 Times
Default re: Advice on designing a 'Sleep Detection System'

Quote:
Originally Posted by nigel92 View Post
@netferakbombay,Our guide is making it look relatively simple. I have no prior knowledge of image processing. I hope he's guiding us in the right direction.
Given that he is looking for things to be done by March 31st, MATLAB looks like the only solution.

MATLAB has bunch of plugins for video processing and feature extraction from frames. Do you have access to any of those in institute or elsewhere?
NetfreakBombay is offline   Reply With Quote
Old 20th February 2014, 20:17   #12
Newbie
 
Join Date: Feb 2014
Location: Bangalore/Goa
Posts: 20
Thanked: 32 Times
Default re: Advice on designing a 'Sleep Detection System'

Quote:
Originally Posted by NetfreakBombay View Post
Given that he is looking for things to be done by March 31st, MATLAB looks like the only solution.

MATLAB has bunch of plugins for video processing and feature extraction from frames. Do you have access to any of those in institute or elsewhere?
I have Matlab installed on my computer. I'll have to purchase the FPGA kit later. A webcam will be used for capture. Any idea of how to make this work in the night?

Nigel
nigel92 is offline   Reply With Quote
Old 21st February 2014, 11:24   #13
Senior - BHPian
 
scopriobharath's Avatar
 
Join Date: Nov 2007
Location: Chennai
Posts: 3,565
Thanked: 1,175 Times
Default re: Advice on designing a 'Sleep Detection System'

Quote:
Originally Posted by nigel92 View Post
I have Matlab installed on my computer. I'll have to purchase the FPGA kit later. A webcam will be used for capture. Any idea of how to make this work in the night?

Nigel
Night vision IR camera can capture the "eye" as a bright white streak /ball

use this as a reference, compared to a sleepy eyelid.
scopriobharath is offline   (1) Thanks Reply With Quote
Old 22nd February 2014, 16:38   #14
BHPian
 
charanreddy's Avatar
 
Join Date: Nov 2013
Location: Bangalore
Posts: 143
Thanked: 138 Times
Default Re: Advice on designing a 'Sleep Detection System'

My 2 cents on this interesting project.

1. In less than ideal light conditions or when there is some shaking, like it is bound to be when you are driving, the images captured from webcam are not like those in your first post. I had faced this problem practically when working on my final year project (Object tracking using Vision algorithms esp. Camshift algorithm) on a mobile robot moving at very slow speed, so I can only imagine how it would be on a fast moving vehicle on uneven roads.

2. Assuming you are able to capture usable pictures, Do check if mugshot/ feature detection algorithms can help. I.E Lets say in 20 to 30 successive frames captured 0.2 secs apart ( 4 to 6 seconds), if for majority of the time, you are not able to find the eyes open in majority of the cases, you can trigger an alarm.

A couple of pointers from one such algorithm I know is

Viola, Paul and Michael J. Jones, "Rapid Object Detection using a Boosted Cascade of Simple Features", Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001. Volume: 1, pp.511–518.

http://www.mathworks.in/help/vision/...ctorclass.html

Hope this is useful.
charanreddy is offline   (1) Thanks Reply With Quote
Old 22nd February 2014, 23:33   #15
Newbie
 
Join Date: Feb 2014
Location: Bangalore/Goa
Posts: 20
Thanked: 32 Times
Default Re: Advice on designing a 'Sleep Detection System'

Quote:
Originally Posted by charanreddy View Post
My 2 cents on this interesting project.

1. In less than ideal light conditions or when there is some shaking, like it is bound to be when you are driving, the images captured from webcam are not like those in your first post. I had faced this problem practically when working on my final year project (Object tracking using Vision algorithms esp. Camshift algorithm) on a mobile robot moving at very slow speed, so I can only imagine how it would be on a fast moving vehicle on uneven roads.
We never thought of this condition. Thanks a lot for pointing it out. Since our final demonstration won't be in a car we can get away with it unless the external examiner reads this thread.

Quote:

2. Assuming you are able to capture usable pictures, Do check if mugshot/ feature detection algorithms can help. I.E Lets say in 20 to 30 successive frames captured 0.2 secs apart ( 4 to 6 seconds), if for majority of the time, you are not able to find the eyes open in majority of the cases, you can trigger an alarm.

Hope this is useful.
I tried to execute some codes from the link you mentioned but i was unsuccessful as i don't have the image processing toolbox installed. Is this addition available free of cost? I have my next class on Monday. At the moment we're stuck at detecting the eyes. Our guide told us to detect whether the eye is open or closed using only 'if else' statements. We haven't yet succeeded.


Nigel
nigel92 is offline   Reply With Quote
Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search


Similar Threads
Thread Thread Starter Forum Replies Last Post
Designing a Home Theatre System Samurai Gadgets, Computers & Software 47 26th February 2014 12:56
world’s first pedestrian detection system bmw530i Technical Stuff 0 23rd February 2010 13:08
Anti Sleep Accessory genesis Modifications & Accessories 18 31st May 2009 16:42
Miles To Go Before I Sleep....... rakesh_raja Travelogues 1 11th October 2008 01:21
Natural music for sleep??? furioussphinx Shifting gears 7 11th May 2006 03:18


All times are GMT +5.5. The time now is 03:25.

Copyright ©2000 - 2017, Team-BHP.com
Proudly powered by E2E Networks