Seam Cerving And Salient Detection For Thumbnail Photos

Image resizing is a process of processing images or images with the aim of changing the size of the image. The most commonly used methods are cropping or scaling. Scaling is changing the size of the image based on the scale. Contents in the image are not considered in scaling. Seam carving often uses energy functionality that is useful as a determinant of the pixel level contained in an image. Seam is a connecting path of image pixels both vertically and horizontally that is passed by a low energy function. Changing the image size using seam carving is considered better than cropping and scaling. However, the seam carving method still cannot protect the object that is considered the most important. In overcoming this weakness, we can use a combination of seam carving algorithm with salient detection. In this research, we will improve the two methods which function as thumbnail maker. The results of the salient detection of the most important areas of the image will be detected and as a reference in resizing the image (seam carving) The dataset uses 200 images. The accuracy value is calculated by distributing questionnaires to 100 respondents and producing an acceptance rate of 78% so that the results are very natural/natural.


Introduction
Currently, many internet users use electronic devices, both mobile phones, computers, and laptops to access web pages that tend to have digital images or photos in them. So that in accessing web pages it is not too large, then creating thumbnails is one way to make image browsing more efficient because the image size is not too large. Thumbnails are images / photos with a size that has a small capacity size of an image with a function only for interfaces on a web page.
Image resizing is a process of processing images or images with the aim of changing the size of the image. The most commonly used methods are cropping or scaling. The function of cropping is to remove pixels within certain limits in an image (Achanta & Süsstrunk, 2009). Scaling is changing the size of the image based on the scale. Contents in the image are not considered in scaling. As a result, the resulting thumbnail photos have not been able to convey the information that is considered the most important.
Seam carving is a method that functions to change the size of the image by adding or removing a carve of several pixels from different digital image elements. Seam carving often uses energy functionality that is useful as a determinant of the pixel level contained in an image. Seam is a connecting path of image pixels both vertically and horizontally that is passed by a low energy function (Yin & Li, 2013). Changing the image size using seam carving is considered better than cropping and scaling. However, the seam carving method still cannot protect the object that is considered the most important.
In overcoming the weaknesses above, it is possible to use a combination of seam carving algorithm with salient detection. In this research, we will improve the two methods which function as thumbnail maker. The results of the salient detection of the most important areas of the image will be detected and as a reference in resizing the image (seam carving). Seam carving will resize the image based on the specified target size length x height.

Salient Detection
In psychology and the field of observation it has been found that (Achanta, Estrada, Wils, & Süsstrunk, 2008), (Rokhim, 2015) salient detection is when we look at an image, our visual system will first quickly focus on one or more "interesting" areas before further exploring the content. the entire image and this area is often called salient detection (Zhang, Gu, & Li, 2013).
Salient detection is useful in many highlevel image processing tasks including image segmentation and object recognition. It can be used as a preprocessing step to reduce search space. One of the important applications of salient detection is image retargeting.

Seam Carving
Semmis a vertical or horizontal path that is connected in each pixel (Avidan & Shamir, 2007). The seam connecting the top and bottom rows of pixels is called a horizontal seam. The number of pixels passed by the seam is equal to the horizontal image height and image width (for horizontal seams). The seam which has the smallest amount of cumulative energy is called the optimal seam. Carving is the process of reducing or increasing pixels that are exceeded by the optimal seam. The seam carving technique is where to remove unnoticeable pixels that are mixed with the surrounding area. The following that can be used as an example in seam carving can be seen in Figure 2.
Seam carving can be used to enlarge and reduce the image, delete objects in the image and others (Avidan & Shamir, 2007) (Sebastian, Santosa, & R., 2016).
Seam Removal In Algorithm 1 it is explained that the stages ywhat is in seam carving is to calculate the energy for each pixel, then generate a list of seams whose order is based on energy. Seam calculation is done vertically and horizontally.
Seam vertical sv of image I with size n x m is shown in Equation 1 where i and col(i) are row coordinates and column coordinates, respectively. (1) During the removal of low-energy seams, important image content is retained during the resizing process. For that, the energy of each pixel can be calculated by the energy function e according to Equation 2. (2) Where and are the derivatives of the row and column. (3) After calculating the minimum cumulative energy value M for all possible seam, seam with the lowest energy s* is found from the lowest value in the last row of the M matrix. The lowest energy is then used for the seam removal process without removing important content in the image.

Dataset
Dataset in this study, 200 images were taken from the internet. With a percentage of 35% of images with objects, 30% of images with landscapes, and 35% of images with faces

Research Design
Furthermore, the process of detecting important areas of an image using saliency detection is carried out. Then the results of the important areas of the salient are used as benchmarks in seam carving. After that, the output image will be obtained in accordance with the expected size. Then analyzed the results of the output image whether it is in accordance with what is expected. (Figure 3)

Trials
At this stage the user will input the image and then do saliency detection using methods based on Lighting and Color (Achanta & Süsstrunk, 2009), Lighting, Color and Map Orientation (Yin & Li, 2013), Lighting, Color, and Contrast (Yan, 2010). The next step is to open the Groundtruth image to determine the initial truth value and then look for the RMSE of Lighting and Color (Achanta & Süsstrunk, 2009), Lighting, Color and Map Orientation (Yin & Li, 2013), Lighting, Color, and Contrast (Yan, 2010). used as a reference for seam carving after that perform seam carving according to the target and pay attention to the most important areas (salient detection) to get the results of the last stage of seam carving images, namely opening the groundtruth of the seam carving image to find the RMSE value of seam carving (Figure 4 & Figure 5). and the validity of the answers generated. The more questionnaires submitted will increase the thickness of the questionnaire, of course this can reduce responses and increase the emotions of the respondents. If this happens, the level of answer validity will decrease due to the appearance of random answers..
The questions in this questionnaire use a rating scale. The type of rating scale chosen is a rating scale with 4 choices (even). On a 4choice rating scale, respondents were not given a neutral answer choice or an intermediate choice which is often chosen by respondents, especially general respondents. Thus, the level of validity of the answers will be better than the rating scale with the number of odd (odd) choices. The specifications for the respondents are a. male or female aged 20-50 years b. Minimum education high school equivalent. Analysis of the results of the questionnaire in assessing the naturalness and naturalness of an image resulting from seam carving. Where the questionnaire was submitted to 100 respondents with 30 different images, namely 10 images containing faces, 10 images with images of objects and 10 images of landscapes. Each image will be assessed for its naturalness and the naturalness of the results of the seam carving process.
Where each question will display 4 answer options: Very Unnatural/Natural, Less Natural/Natural, Fairly Natural/Natural, Very Natural/Natural (Rokhmah, Zaman, & Pranoto, 2020). So that the resulting answer as in Table  1.
So that the criteria for interpreting scores based on intervals are obtained, namely:  • Numbers 50%-74.99% = Quite Natural/ Natural • Numbers 75%-100% = Very Natural/ Natural Overall, from the 30 images carried out by the questionnaire, interval and average values can be generated, namely in Table 2.

Conclusion
From the research that has been done by the author, it can be concluded that: From the test results using 3 different salient detection methods that to find the most important areas with salient detection, the method with better results based on RMSE is to use the salient method which uses features based on lighting, color and map orientation, referring to the Image Re-targeting Based paper. on the Novel Saliency Map, Kai YIN and Ying LI (Yin & Li, 2013).
Based on the results of a questionnaire with 30 images resulting in an acceptance rate of 78%, so that it can be seen from the score interpretation criteria based on intervals, the results are Very Natural/Natural. In making thumbnails, you can use the seam carving algorithm combined with salient detection.

Suggestion
In further development, seam carving can also be implemented using video input instead of images.