3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Ed. 36 Vol. 9 N.º 4 Diciembre 2020 - Marzo 2021
95
DEVELOPMENT OF SMART PAINTING MACHINE USING IMAGE
PROCESSING
Atif Saeed
Faculty, Department of Mechatronics Engineering, SZABIST, (Pakistan).
E-mail: m.atif@szabist.edu.pk ORCID: https://orcid.org/0000-0003-4369-2388
Hussain Muslim Mithaiwala
Student, Department of Mechatronics Engineering, SZABIST, (Pakistan).
E-mail: hussain.muslim53@gmail.com ORCID: https://orcid.org/0000-0001-6284-6237
Ammar Iqbal Hussain
Student, Department of Mechatronics Engineering, SZABIST, (Pakistan).
E-mail: ammararsi48@gmail.com ORCID: https://orcid.org/0000-0003-2031-8393
Tulsi Kumar
Student, Department of Mechatronics Engineering, SZABIST, (Pakistan).
E-mail: tulsikumar199600@gmail.com ORCID: https://orcid.org/0000-0002-8848-0533
Recepción:
04/09/2020
Aceptación:
02/10/2020
Publicación:
14/12/2020
Citación sugerida:
Saeed, A., Mithaiwala, H.M., Hussain, A.I., y Kumar, T. (2020). Development of smart painting machine using image
processing. 3C Tecnología. Glosas de innovación aplicadas a la pyme, 9(4), 95-119. https://doi.org/10.17993/3ctecno/2020.
v9n4e36.95-119
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Ed. 36 Vol. 9 N.º 4 Diciembre 2020 - Marzo 2021
96
https://doi.org/10.17993/3ctecno/2020.v9n4e36.95-119
ABSTRACT
Painting is a process in automotive industry that takes about 10% of the total time consumed per vehicle.
Currently, the painting lines in several automotive industries like Toyota motors or General motors
is largely being automated after implementation of robotic arms. However, according to the study
done, this automation is restricted to paint main body or chassis only, the coating and painting on small
vehicle parts is still done manually. This paper represents the study conducted on local Toyota motor
plant located in Karachi where the need of automating the small parts painting line evolved to increase
production. The designed system is based on 3axis gantry mechanism which locates the position of parts
to be painted on moving conveyor through computer vision, reach the coordinates, paints autonomously
and forwards it to buer station for drying. The computer vision is introduced to minimize the human
interference and make the line semi-autonomous. The system was tested for correct image processing
and at the end algorithms was correctly identifying and locating 90% of the parts. Moreover, the system
was tested for one week in a laboratory environment and it was observed that production rate was
increased to 2% as compare to their previously available data.
KEYWORDS
Computer Vision, Autonomous Painting, CNC, Automobile, Mechatronic System, Python.
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Ed. 36 Vol. 9 N.º 4 Diciembre 2020 - Marzo 2021
97
https://doi.org/10.17993/3ctecno/2020.v9n4e36.95-119
1. INTRODUCTION
Today, the modern industrial revolution is on its way towards advancements. The industrial revolution
is taking over the entire world briey. One of the major goals of today’s industrial revolution is to
adopt automation. Almost in every sector such as military, agriculture, automobile, manufacturing
and design and development etc. Presently, automation is widely being implemented and used in the
industrial sector all over the world, this is done in order to increase the eciency and productivity of
the manufactured goods. The automobile sector is also using automation to perform various task in the
cars body manufacturing, assembly and paint shops in order to increase their per day productivity and
eciency of the cars. But however, the industrial automation is not implemented and used in Pakistan
at its maximum pace. Thus, a lot of task are done manually, this aects the eciency and productivity
of the produced parts. In Pakistan, especially in the automobile sector, painting procedures are not
fully carried out by robots, instead paid labors (humans) are also employed in painting small parts such
as mirror covers, mud aps, door handles, spoiler and fog lamps etc. And also, automatic painting is
adopted only by factories and industries which produce large lot sizes of same type/variant of parts
(Saeed et al., 2019).
Thus this paper discusses about the possible feasible solution which will help shift the small scale industries
and the automobile industries to implement modern industrial automation by placing robots in their
production line in order to increase their productivity and eciency of produced goods.
Currently, Pakistan is not able to completely not able to adopt the industrial revolution along with the
modern automation as because the initial capital cost is way too much for which small industries can’t
invest in it. The other major reason is that most of the robots can paint only one part variant at a time,
thus industries not producing large lot sizes of same part variant are unable to get automatic painting
being carried out in their industry. Another problem caused by this is the human health issues, as painting
process contains release of harmful chemical fumes which ultimately aects human health. Also human
painted parts are sometimes not up to the mark as the paint coating are uneven, thus the parts are not
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Ed. 36 Vol. 9 N.º 4 Diciembre 2020 - Marzo 2021
98
https://doi.org/10.17993/3ctecno/2020.v9n4e36.95-119
passed from quality check departments which results in reducing the productivity as well as eciency of
the parts produced by the industry. Thus, all these reasons aect the productivity and eciency of the
industrial produced goods (Amin & Saeed, 2018).
Our idea is to introduce a Smart Painting Machine which will be capable of painting various geometry
parts at a time. By this our aim is to introduce the concept of automatic painting in small scale industries
who cannot aord costly automatic painting machines within their vicinity. We basically derived the
idea of automatic painting from the automobile industries where still small parts such as mud aps, door
handles, spoilers, fog lamps etc. are painted by labors.
The key feature of this machine is basically its programming part which waves o the requirement to
burn/run dierent program for dierent geometrical part. The programming embedded with the smart
painting machine is basically based on image processing. A camera is mounted in the workspace of the
painting machine which takes the real-time image of the part and then the image is processed. After
processing the image, its geometry is detected by applying edge detection and contour detection. This
results in identifying the object geometry by which the object coordinated are detected. Thus the part is
painted as per its geometry. This features allows small lots of various geometrical parts to be painted with
a single machine and single program. Thus, it’s very benecial for industries as it saves a lot of capital
investment.
As far as human health is concerned, this machine follows all the safety protocols. The painting workspace
is enclosed by a glass/acrylic frame house to eliminate the chances of spilling out paint particles as well
as escape of any painting chemical fumes. Thus a lot health issues like coughing, sneezing, nausea and
breathing issues are eradicated.
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Ed. 36 Vol. 9 N.º 4 Diciembre 2020 - Marzo 2021
99
https://doi.org/10.17993/3ctecno/2020.v9n4e36.95-119
2. LITERATURE REVIEW
Authors, in this paper, plans to introduce a process for advance mechanize spray painting of unknown
parts. The machine made by this experiment will be very useful for painting of unknown shape. The time
consumed by this machine is very less as compared to manually, (hand) painting. It will, save expenditure
such as labor cost and the total cost of painting any jobs (Swarakar et al., 2018).
Thakar and Vora (2014) in his paper gives essential knowledge about mini and large scaled industries
manufacturing parts are painted for protecting from rust, so the spray method consumes large amount
of time and paint which required the workers which are skilled emerged with the application. Robotic
painting techniques is not applicable for large eciency so the rise in such method have to be made
which is aordable, have accuracy and precision, consumes minimum time for the coating of the part so
objective has to developed in such a manner that the mechanism which coat the part with the dipping
and baking process having semi-automated techniques which is up to the required mark and which can
be valuable for mini and large scale factories (Thakar & Vora, 2014).
Author in this research highlights some key features through the system test, it is fact that the design
of intelligent robot have many advantage not only has good painting eect but also has high eciency
which can only at least 2-6 minutes to draw the simple cartoon images and also take no more than 15
minutes to draw complex portrait images. We also provided an illustrative example to show our required
results (Feng et al., 2017).
Abdellatif (2012) in his research describes the design, construction and working of an Automatic wall
painting robotic machine. This visionary and remarkable design of a robot which is movable to be used
for painting interior walls of residential building or oces has been described. Robot has a roller that is
fed with liquid paint and keeps contact with the wall surfaces. The robot has advance option that helps
the roller to scan vertically as well as horizontally to the painted walls. The robot has advance technology
that can adjust itself in front of the wall (Abdellatif, 2012).
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Ed. 36 Vol. 9 N.º 4 Diciembre 2020 - Marzo 2021
100
https://doi.org/10.17993/3ctecno/2020.v9n4e36.95-119
Naticchia et al. (2007) in his research, shows that mechanical painting can be not only be done to upgrade
production and allows quality checking. The robotic arm application with high precision and accuracy is
required. An automatic system to convert to the normal coordinates of the liquid colors to be reproduce
the moveable speed of the robot’s end tool and valve opening and closing end of the mixing board. The
maximum work shall be probably required to get high resolution (Naticchia et al., 2007) hence also the
productivity of the construction industry must be improved, while preserving its labour from hazardous
job sites. Such requirements can be accomplished by the adoption of robotized products, which, however,
need to be quickly developed and marketed. In this paper, rst the issue of a new miniature laboratory
for developing lightweight and well-coordinated robotized systems is pursued, then a novel robot device
for high quality multi-colour interior wall painting carried by a robot arm is developed and successfully
tested. Thanks to the new 1:6 scaled down laboratory and its six degree of freedom robot arm on an
hexapod for horizontal moves, we tested the opportunity to introduce also in the building sector miniature
robots that can change the ergonomics standardly adopted by construction workers. It is analyzed how
and why switching from full size to miniature robots is convenient in construction. In addition, a new
system adding further features to robotized painting has been conceived. Our new multi-colour spraying
end-tool was developed and xed on the robot arm, in order to be able to reproduce coloured artworks.
Finally, a methodology to reproduce colours from digital format of artzoorks is presented, showing
how accurate and ecient is this new robotized spraying device. miniature painting robot, scaled down
laboratory, multi-colour spraying end tool. Ata and Eleyan (2017) in their research present his work on
articulated robots like these robots are widely re-known by basically automobile company commercials
and robot dance application. SCARA (Selective-Compliance-Articulated-Robot-Arm) robots are also
re-known for their usage and proliferate in industries from 1970’s. Two kind of robot articulated and
SCARA robot’s combination to gather linear and rotary motion accomplishing in formation for complex
tasks (Ata & Eleyan, 2017).
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Ed. 36 Vol. 9 N.º 4 Diciembre 2020 - Marzo 2021
101
https://doi.org/10.17993/3ctecno/2020.v9n4e36.95-119
In this research paper goal was to learn the system for coating and painting tasks carried out in automobile
repair and then change manual painting by robot painting. So the most important study, was for skilled
spray gun handling for automobile repair painting which were observed and compared with those with
little or no experience. The spray gun movements of the experts were characterized by longer length,
longer time, higher speed, and narrower swing range, compared with the non-experts. The results were
collected and accordingly the spray gun movement was set (Ikemoto et al., 2015).
Authors in this paper presents a research on Image Processing that can be directed to Machine Learning
and the process of computing can identify pattern of high diverse parts. Machine learning is very close
and like computing statistics that consist of spam lter optical character identication searching engines
and computer vision. Their extensive arrays of knowledge observed (algorithms) to reduce destruction
noise such linear lter of Gaussian-based algorithm. Algorithms can remove certain kind of grain noise
destruction from a picture. Because every pixel of picture in setting to mean values in its environment
the normal variations tested by the grain are reduced (Wiley & Lucas, 2018).
Authors in this project through his studies, successfully identied the part from the background picture
used for color process is required to remove the background by 1st ler grayscale ltering is the second
step and nally by Circular Hough Transform (CHT) and binary testing for part that is in circular object
detection. Using of color processing is used as it is powerful process to identify the part as it is in normal
color process it has a lot of information as human eyes can do. For the grayscale lter it lters the (pixel
and smoothness) the picture to the edge clear. In last CHT is required to detect the parts which are
circular and total number is displayed (Hussin et al., 2012).
Authors in this research paper formulates that it is not possible to consider a single process for all type
of images, nor can all process perform well for types of image. The background subtraction process
identify parts with noise destruction and output is not accurate and precise. The object behind object
is not recognized. During identication of part when any other thing come before the part problem
occurs. The image cannot be recognized if the position of camera is not accurate and object in picture
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Ed. 36 Vol. 9 N.º 4 Diciembre 2020 - Marzo 2021
102
https://doi.org/10.17993/3ctecno/2020.v9n4e36.95-119
is not snapped properly (Jain & Chadokar, 2015)Object recognition is a work of searching a selected
object in an image or video sequence. Object detection plays a key role in image processing, It helps in
searching of any particular object Object recognition is use to detect a particular object from a series of
other objects, sometimes shadow and background images becomes problem in detection. Skull detection
technique is use resolving such kind of problems. Here we deals with dier sort of object detection
techniques and modes of multiple object detection for a image.
The authors, in this paper, through his knowledge and work proposes an algorithm that has been proved
to meet the requirement of object detection without using the color feature in an automatic robot. The
proposed algorithm specially relies on two main process that emphasize on shape identication and
feature extraction that follows.
The rst method is edge detection and line-oriented method to performing contour extraction, which
results in object detection in very less or no time. Next, the second method is a geometric moment that
captures and computes the global features of the objects. Both the process is well assured in image
processing, however, a mixture of both is a novel method in this study and has been proven to precisely
detect static and moving object under illumination variety (Dewi et al., 2019)such as the lengthy process
to calibrate color, color fading, and others. Nonetheless, the need of such application that does not
necessarily rely on color information has seen a hike due to the mentioned issues. In fact, some of the
desirable solutions are those that take less computation time, as well as those that provide higher accuracy
and scalability for a large number of objects in a scene. One application that requires such solution is in
a game playing by autonomous robot. This paper suggests a novel patch carried by autonomous robots
with relevant detection algorithm using contour detection and geometric moment without using the
color feature.
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Ed. 36 Vol. 9 N.º 4 Diciembre 2020 - Marzo 2021
103
https://doi.org/10.17993/3ctecno/2020.v9n4e36.95-119
3. METHODOLOGY
3.1. DESIGNING PHASE
3.1.1. HARDWARE DESIGNING
So initially, it is not recommended to go towards hardware fabrication directly. It is more desirable to
rst virtually design the hardware on a CADCAM Software, such as SolidWorks. Mostly, the thinking
and fabricating part does not go hand to hand simultaneously as it’s necessary that what we think can be
implemented and fabricated practically, this leads to loss of time and capital. So, in order to check our
design ideas and its practical fabrication feasibility, we do the designing work rst.
So, in the designing phase, we rst started with our conveyor. First the conveyor frame structure of
dimensions 70in x 20in x 3in was made. To understand it, it is basically like a table structure. Now,
in order to run make the conveyor bed, we attached 2 rollers at each end of the conveyor frame. By
attaching two rollers, it was observed that the conveyor belt be stable while moving as when the part
comes in the middle of the conveyor belt while travelling, the belt will sag downwards due to no roller in
the middle and this will cause irregular movement of the conveyor belt. Thus, we added a third roller in
the middle of the conveyor frame.
We, now, added a white rubber belt of 2mm thickness over the rollers to form a conveyor bed. Three
double slotted pulleys of were mounted on each on the three rollers in order to interconnect the rollers
with each other so that movement of the rollers are in sync. These pulleys are interconnected 2 rubber
V-Belt and the third belt of size relates to a pulley and with a D.C Geared Motor of 30Nm torque (24V-
3A) in order to drive the conveyor. The part to be painted will be placed on the conveyor which will
help in transporting the part to and from the painting work area. The conveyor design is given below in
Figure 1.
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Ed. 36 Vol. 9 N.º 4 Diciembre 2020 - Marzo 2021
104
https://doi.org/10.17993/3ctecno/2020.v9n4e36.95-119
Figure 1. Conveyor Design.
After the completion of the Conveyor Design or the Part Transporting Mechanism, we started to design
the CNC Mechanism which is the Painting Mechanism of our Machine. The overall size of the CNC
mechanism is 24in x 20in x 24in. Each of the 3 axes of the CNC machine is made of square shaped
2.5”x1” mild steel pipe. The main axis mechanism is formed of a ball screw of length 500mm and 8mm
size of double threads. Each of the two ends of the ball screw is connected in the 8mm bearing which
is mounted inside bearing housing.
Now, we will screw an 8mm double threaded nut over which an aluminum cube is xed which acts as
a traveler. All these things are xed to form a single axis structure. We replicate this structure four time
to form X, Y and Z axis individually (X axis is made from 2 of these structures). We mount individual
Stepper Motor on each axis in order to rotate the ball screw so that the aluminum cube mounted over
a nut move linearly back and forth. Each axis motion is limited/restricted by attaching a limit switch
at its both ends. This helps in restricting the axis motion in forward or backward direction, so all the
3-axis does not collide with each other. The painting spray gun is mounted on the Z-axis of the CNC
Mechanism. The Design of CNC Mechanism is given below in Figure 2.
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Ed. 36 Vol. 9 N.º 4 Diciembre 2020 - Marzo 2021
105
https://doi.org/10.17993/3ctecno/2020.v9n4e36.95-119
Figure 2. CNC Mechanism Design.
This CNC Mechanism is mounted on the conveyor in the middle, thus the area under the CNC
mechanism is the painting working area. Ball screws are used to move the 3-axis of the CNC mechanism.
These ball screws are rotated with the help stepper motor and stepper drives are also used. The painting
spray gun is mounted on the Z-axis of the CNC Mechanism.
Figure 3. Smart Painting Machine Complete Hardware Design.
3.1.2. ELECTRONIC CIRCUITRY DESIGNING
Apart from the hardware of the machine, the Brain which will help in driving the functionality of this
machine is the Electronic Circuitry. Basically, the electronic Circuit of this machine needs to be designed
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Ed. 36 Vol. 9 N.º 4 Diciembre 2020 - Marzo 2021
106
https://doi.org/10.17993/3ctecno/2020.v9n4e36.95-119
exclusively for itself. The readily available circuits can’t be integrated with one another to drive this
painting machine, instead its circuit needs to be designed. The main task required by the circuitry is to:
1. Drive and Control the motion of the Conveyor.
2. Drive and Control the motion of the Painting Mechanism (i.e. 3-axis CNC Mechanism).
3. Assist the Image Processing Program in execution so that the part Geometry can be identied, and
it’s coordinated can be extracted in order to move the painting mechanism to perform the painting
task.
Thus, in order to accomplish this task we chose Raspberry Pi Microcontroller which will control the
entire circuitry. Raspberry Pi will be powered by a 5V supply input which is fed from a 24V Power
Supply after connected a buck converter in series in order to step-down the voltage. Also, the conveyor
motor will be powered by this power supply and its programmable controlling will be handled by the
switching of a relay which is connected to the raspberry pi microcontroller. The Raspberry Pi Camera
Module will be connected to the microcontroller which through its assistance will capture the image
of the part which will enter the workspace to be painted. Also the Image Processing Software, which
is developed on OpenCV to detect the part, identify it and calculate its geometrical coordinates will
also be executed by this microcontroller. Also, after the extraction of the parts center coordinates, these
coordinates will be via Encoder and TTL IC will be fed to Arduino.
The Arduino Microcontroller will take in the centroid coordinates of the part (X, Y, Z) and then will
drive the painting mechanism motors in order to paint the part. The motion of the axis is limited by
Limit Switches.
The Designed Circuit for Smart Painting Machine is given in Figure 4 and Figure 5 below.
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Ed. 36 Vol. 9 N.º 4 Diciembre 2020 - Marzo 2021
107
https://doi.org/10.17993/3ctecno/2020.v9n4e36.95-119
Figure 4. Design of Circuit 1.
Figure 5. Design of Circuit 2.
3.1.3. SOFTWARE DESIGNING
The heart of this Smart Painting Machine is its Software. The software is basically termed as heart
because the sole purpose which makes this machine smart is the image processing program. We thought
of applying image processing in this machine as to get rid of uploading new programing from every
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Ed. 36 Vol. 9 N.º 4 Diciembre 2020 - Marzo 2021
108
https://doi.org/10.17993/3ctecno/2020.v9n4e36.95-119
new part and forbid the idea of keeping/placing the part to be painted in a xed place or to be xed
in its respective jigs. Image processing will help in determining the part orientation on the conveyor
bed and will calculate the part coordinates and will command the motor to move the axis respectively
in order to paint it. To implement this idea, we had to rst check that whether his idea is working or is
practically feasible to implement on a machine to calculate the part coordinates on real-time basis. Thus,
for developing, testing and nal implementation of this program, we chose to do this task using Python.
The camera which will be taking the part image will be mounted over the workspace. Once the part
enters the workspace, IR sensor will indicate the microcontroller to stop the conveyor and then the part
image will be taken and processed and then the part will get painted and then the conveyor will move,
and the part will depart the workstation. So, to implement the software idea, we made program which
works as follows:
First the part image is read by the program. After reading the image, lters of Erosion (process which
removes pixel from boundary of the object in an image) and Dilation (process which adds pixel to
boundary of the object in an image) are implemented. Then the object in the image is subtracted from
its background and then thresholding is applied on the image. After this Canny Edge Algorithm (which
detects edges of the object with noises suppressed at the same time) is applied and then the Counter
detection is applied (which joins the curve of all the continuous points along with boundaries of same
pixel intensity).
This enable the program to calculate the center of the object and its dimension. After this the coordinated
of the image is found. Then the motor steps are calculated, and this is passed on to the XYZ Algorithm
which then controls the motor movement of the CNC axis. The below Figure 6 shows the Software
design owchart.