Laidlaw Research: OpenCV and Deep learning

Welcome back to my blog! I'll be talking about weeks 3 and 4 of my Laidlaw research project where I delved deeper into some more image processing in OpenCV and began researching my final approach for a CFU detector. I hope you enjoy the read!

Jul 02, 2021

Joe Linogao

Software Engineer, LegitFit

Liked by Trinity College Dublin and 1 other

Week 3: Utilising better Pre-processing Methods

During week 3, I decided to work on image pre-processing. The main problem with the images from the last blog was that other features of the images, such as the background and plate, were still being shown in the image. However, with the help of computer vision, I was able to detect and remove the background from the image, keeping the contact plate intact as shown below.

I also tried fixing up the contact plate, blurring any drawn lines that were present in the image. However, I discovered that there was data being lost due to the blurring algorithm. Notice how the smaller CFUs are being lost after processing.

Given the time constraints, it seemed like it would be a wasted effort to continue with the thresholding method (however, I did keep the background remover). This was around the time my supervisor introduced me to his colleague who had a background in Deep Learning, particularly regarding object detection. From there, a meeting was set up and I was on my way to my supervisor's company space.

Week 4: Colony Object Detection

Week 4 started with the meeting with my supervisor's colleague. We talked about what I have done so far with fast.ai and the dataset that was available to me. We then decided on identifying individual CFUs using object detection. Object detection can be seen as a subset of deep learning, with the following steps:

Image classification: What is in this image?
Object localization: Where are the "objects" in this image/creating bounding boxes.
Object detection: Identity what each "object" in the image is.

We found this to be the best approach as it shouldn't rely on the quality of the threshold mask since it doesn't need one at all. The only problem is, the best way to do object detection was through Keras since that was the program they were most familiar with and could help me with. No matter though, I'm always up to learning new things (famous last words).

 
Aside from learning a new deep learning library, I also had to create a new annotated dataset, as well as research suitable models/weights for transfer learning. Here's one of the images that are a part of my dataset so you can get a feel of what I am working with.

Oh! And, I also got to control some robots too which was pretty cool!

So that's it for my update blog! Again, I hope you all enjoyed the read! To be honest I'm starting to feel the pressure with this research. I'm essentially learning as I go regarding deep learning, but it's very rewarding when stuff works. I'm also a bit more confident with my communication and networking after that meeting, so that's a plus!

Remember, if you want to keep up to date, follow my Instagram, @joeceengineering!

Until next time, thanks again for reading!

Joe Linogao (He/Him)

Software Engineer, LegitFit

Hello! My name is Joe Linogao (he/him), and I am currently a Software Engineer at Akara Robotics and Engineering with Management graduate from Trinity College Dublin. I became a Laidlaw Scholar for TCD on April 2021 with my research proposal, "Automated Procedure for the Microbial Analysis of Contact Plates."

I'll be honest, I just copied and pasted my LSAB bio because I'm lazy to switch the paragraph to first person lol

"Joe is an Engineering with Management graduate from Trinity College Dublin, a Software Engineer at LegitFit and an active board member for the LSAB. His journey from mechanical engineering to full-stack software development highlights his adaptability and passion for technology, with his key interests being the startup and Software-as-a-Service space.

Joe's love for creating simple solutions and fun experiences, video games and music drives his innovative approach. His past experiences include research with the Laidlaw Foundation and working as a computer vision engineer at the leading robotics startup from Trinity, Akara Robotics. An ambitious and fast learner, Joe also enjoys Dungeons and Dragons and has a strong presence in content creation, including music and digital media."