How to make visual regression testing with robot framework
How to make visual regression testing with robot framework ?
Robot Framework is a wonderful multi purpose testing framework but it lack of natively visual regression testing, so it will not help you to understand if your web interface has something wrong due to difference in a baseline of pre taken screenshot.
Robot Framework anyway permits to easily generate other testing capabilities adding your hand made keywords maybe loading pure python scripts. Writing this article I know only two ways to try to obtain visual regression with robot framework . You can obtain a great result with applitool, but applitool is a expensive tool and maybe you want try to obtain the same result for free. There are a lot of tools that permits to you to obtain visual regression testing but no one is ready for robot framework, so I’m asking for your help to obtain a result together.
The nearest result is given by robotframework needle https://github.com/laurentbristiel/robotframework-needle that use pure python needle http://needle.readthedocs.io/en/latest/ to make the visual regression testing. Robotframework-needle anyway use a strange approach to the problem, and it hangs : as you can understand taking a look at this code, https://github.com/laurentbristiel/robotframework-needle/blob/master/test-example.robot , the library runs pure python test and wait for cli results, trying to understand if screenshot comparison is good or not by this. This can be a good approach.
I’ll start the research searching for visual regression tool and i found this awesome list in www.elementalselenium.com
Name | Platform | Programming Language |
Applitools Eyes | Selenium & Other | All |
Fighting Layout Bugs | Selenium | Java |
Selenium Visual Diff | Selenium | Java |
CSS Critic | Other | JavaScript |
Gemini | Selenium | JavaScript |
Grunt PhotoBox | PhantomJS | JavaScript |
PhantomCSS | PhantomJS & Resemble.js | JavaScript |
Snap and Compare | PhantomJS | JavaScript |
Specter | XULRunner | JavaScript |
WebdriverCSS | Selenium | JavaScript |
FBSnapshotTestCase | Other | Objective-C |
VisualCeption | Selenium | PHP |
dpdxt | PhantomJS | Python |
Huxley | Selenium | Python |
Needle | Selenium | Python |
Wraith | PhantomJS | Ruby |
Wraith-Selenium | Selenium | Ruby |
So i will start taking a look at dpdxt, Huxley and Needle.
See you soon.