Drag and drop files and flash automation in Selenium using Sikuli

If you are looking for a solution to automate files drag and drop and Adobe flash content in Selenium, you are in the right page. In this post I will familiarize you about Sikuli. Sikuli automates anything you see on the screen. It uses image recognition to identify and control GUI components. It is useful when there is no easy access to a GUI's internal or source code.

About Sikuli:

Sikuli is an open-source research project originally started at the User Interface Design Group at MIT.

How to setup sikuli in Selenium

It's quite simple. You have to download Sikuli_setup.jar from Sikuli project page. Here is the link to download Sikuli. Click here.

Once the download is finished, copy the jar file to any folder. Click the sikuli-setup.jar to start the setup process. Once you double click the JAR file, you will observe two extra files been created, a batch file and a log file. 

Click the prompt shown for your information and proceed. Next, you will observe a screen with few options. To automate using Sikuli along with Selenium in Eclipse, you need to select Options 1,4,6 only.

 Click okay to proceed. Once the setup is finished, you will get two JAR files sikuli-ide.jar and sikuli-java.jar. One is the Sikuli IDE and other is library file you need to add in eclipse project library to write sikuli scripts in eclipse.

Setup environmental variables for Sikuli

You need to add a system environmental variable %SIKULI_HOME%

Since all setup is finished, we may proceed with scripting. Hope you have added Sikuli library file in Eclipse. Now create a java class for Sikuli and write script as below. You may call this method whenever required.

Code for flash automation:

package newpackage;

import org.sikuli.script.FindFailed;

import org.sikuli.script.Screen;

public class Photocapture{

public static Screen screen;

public static void saveimage() throws FindFailed, InterruptedException{

screen =new Screen();

Thread.sleep(1000);

screen.click("images/allowcamera.png");

Thread.sleep(5000);

screen.click("images/captureimage.png");

Thread.sleep(2000);

screen.click("images/saveimage.png");

Thread.sleep(2000);

}

}

Like this you may use functions for drag and drop files. All you need is the image of source file and image of target.

In order to drag and drop files, use below statement:

screen.dragDrop("source image path", "target image path");

Always remember Sikuli automation will work when the given image is visible in your screen.

 I would like to thank Sikuli project team for their wonderful work :)