Raspberry Pi Camera Images - I made a mistake!

When I set-up the Raspberry Pi camera to take pictures every 2 minutes (see my previous post), I did not check up on the files to see if they were saving properly. My goal was to leave the Pi running for at-least 10 days to get a large amount of images to work on some image processing and convolutional neural networks.

When I did my initial test for 1 days I calculated my image file size to be very small (max of about 200kB). So, if there a photo every 2 minutes, that's about 720 photos a day and 7200 photos in 10 days. Which is only 1.4-1.5GB of space - not a problem. Since my Raspberry Pi zero was did not come with WiFi I didn't bother looking at the progress as I didn't want to disturb the camera.

So where did I go wrong? When I did my first test I had a small Python script which took a photo every 5 minutes, but since the python script tends to leave the camera on as long as it is running, I decided to write a small shell script which can also do the same thing and I scheduled the script from Crontab to run every 2 minutes, but I did not check on the new file size. Apparently running the camera this way actually saves an Image of roughly 2.5MB instead of 200kB - Oops. The rest is simple, my Raspberry Pi exceeded by initial estimates by the end of Day 1 and the file system crashed at the end of Day 2. However, I did manage to extract the files from the SD card.

But turns out this is not the reason why my file system crashed. At the end of day 2 I only had about 2.5GB of space on an 8GB SD card. I knew there was no way my files could have exceeded this limit. So why then? When I did some reading I realized that this was to do with the block size. The block size on my Raspbian OS is about 4096 bytes, which means that any file is saved on fixed blocks of 4MB. Whether my image was 200kB or 2.5MB it would have taken up 4MB to save the file. So, looking at it this way it can be easily seen that to save my 2 days worth of images it had actually taken close to 5GB of disk space.

So technically, this attempt would have been a bust either way. Time to look in to a new method of saving files.

Back to Basics: Building a Buggy

Although I built a handful of various types of buggies during university, I have always wanted to build another one. There is something about putting together a buggy and watching it work that is very satisfying.

I recently came across one of my old drawings from high-school of a buggy that I had wanted to build but never got around do it. So I decided to collect up some of bits and bobs and put together a very rudimentary buggy just to make my lock-down weekend a little bit exciting.

From what I remember from when I was drawing this I wanted a bunch of features on it, like;

• Pull out control panel
• Automatic docking for charging
• Use as balance-bot and
• Line following

Kivy based manual image classification helper

 Previous post: Raspberry Pi camera time-lapse

I currently have a Raspberry Pi camera set up near my window taking a picture of the weather outside my house. Next I have to classify these images to meaningful categories if I am to do anything with this data. This is going to be a very tedious task. So to make things easier for me I built myself a helper using Python and Kivy.

GitHub repo here

What is RPA and why you need to get onboard right now!

Have you seen mundane and repetitive tasks at your office being automated? (It is very likely that you have!) Then you have already experienced RPA or Robotic Process Automation. There is no point in avoiding it. The technology is here now to make it happen and if you don't do it, someone else will, and this is the reason why you need to embrace it.

Photo by Photos Hobby on Unsplash

RPA is a type of business process automation. It's all about improving your productivity by freeing you up from doing repetitive work that benefits both you and your company. Whether you are a sales associate, manager or engineer there will be a part of your job that can be automated to help you focus your time and energy on what is important. This can even be related to your personal life.

Scraping Gold

 For a future project, I wanted to try and get the history of the price of gold. I did a bit of scouting and landed on the history page of goldprice.org. However, the data was in a chart and I wanted to extract it to a table.

To do this, I wrote a very short web scraping tool which scrapes the data off the chart and formats the data points as a numpy array on python.

The GitHub code can be found here.

Raspberry Pi Camera Time-lapse

 Previous Post: Setting up your Raspberry Pi Camera

Once I set-up my Raspberry Pi camera, I wrote a simple script to take a picture from my camera every 2 minutes.

Setting up a Raspberry Pi Camera

I had a small generic Raspberry Pi Camera Rev 1.3 lying around in my parts box for some time and I have always put aside working on it. So I decided to finally take it out and one of my Raspberry Pi zeros to test out the Raspberry Pi Camera.

Scraping Daraz.lk

I have always wanted to learn how to write a code for a web scraper and I finally got around to doing it while at home during lock down. Here is the process on how I got started.

GitHub site: https://github.com/iamJohnnySam/WebScraping

I came across this helpful page on towards data science and got started with learning more about the web features on daraz-Mall and how to navigate them with Beautiful Soup. Having built my initial confidence from reading some more web pages I wanted to scrape daraz.lk/daraz-mall and quickly realized I bit far more than I can handle. The web scraping was successful but the site returned blanks. This was probably because I was using Googles Colab environment to execute my code. 

I next decided to take this one step at a time.