Optical Character Recognition and Computer Vision Technology in UiPath

SINEM BATMACA | TEST SPECIALIST

Robotic Process Automation (RPA) can be basically referred to as software solutions that automate routine and repetitive processes. RPA is considered as a software that has a high impact on Return on Investment (ROI). Demands for RPA systems are increasing, as businesses have witnessed that RPA systems improve business performance. For this reason, many new RPA vendors have entered the market in recent years and among RPA software vendors, UiPath is the market leader.

Undoubtedly one of the most useful and versatile tools of the UiPath platform is the Record feature. Instead of designing process step by step, it enables design the process in the process performed by the user. The software robots tracks these clicks and actions on the screen and converts these steps into a workflow.

However, the record function struggles with distinguishing text fields and buttons, especially in Remote Desktop Applications such as Citrix. When recording in these environments, the entire application window is viewed as a button on the robots side.

In this context, Optical Character Recognition (OCR) technology is one of the solutions used to overcome the automation difficulties in remote environments. During process recording, a UiPath user can select an OCR machine on the UiPath Studio, select the text to be automated in the window, and allow the robot to find the text every time. The fact that the text is in a different place is not an obstacle for the robot to operate. Advanced character and image recognition software is one of the main reasons why RPA is so successful.

On the other hand, automating virtual desktop environments such as Citrix, VMware, VNC and Windows Remote Desktop with high reliability has been one of the challenges of OCR technology. The possibility that minor changes in the UI could disrupt automation leads to reliability and maintenance issues.

The image matching algorithm is prone to errors due to the possibility of changes in the appearance of the target elements. Computer vision ensures that these challenges are overcome with high reliability.

Computer vision technology is basically a feature that allows robots to “see” the screen, identify elements without the need of selectors, and image matching.

This feature, which enables robots to recognize user interfaces like human, is provided by an algorithm developed using a combination of Artificial Intelligence (AI), Machine Learning, Optical Character Recognition (OCR), text fuzzy matching technologies and a                         multi-anchoring system.

In addition to virtual desktop environment, Computer Vision can be used to identify elements in a variety of scenarios where traditional UI automation methods are challenged, including SAP, Flash, SilverLight, PDFs and even images.

Unlike traditional image automation, UiPath’s computer vision feature is not based on image matching. This makes it highly resistant to interface changes, including color, font, size and resolution changes. Computer Vision eliminates the dependence on selectors while maintaining workflows which known by RPA developers. The increase in the number of visible display elements makes further automation possible.

With the development of such technologies, we can say that robots have become more insightful in recent years, which proves that we are one step closer to software automation robots that can make complex decisions.

 

Resources

  1. https://www.uipath.com/blog/what-a-robot-sees-using-ocr-in-rpa
  2. https://www.uipath.com/blog/introducing-new-uipath-ai-computer-vision

 

Please follow and like us:

JMeter Usage

DAMLA IYILIKSEVER | BUSINESS ANALYST

In my previous blog post I’ve mentioned software test tools. Now, I aim to promote you to you’re your own tests on your web applications by giving basic information about JMeter. JMeter is made by 100 % Java codes and you should have related libraries to be able to use it as I specified in my previous post. It provides you to make your web applications’ performance, load, and functional tests. Let’s start to examine JMeter with its basic features and the involved contains.

  • Thread Group: It is the first step of each test group and the all other elements of test plan should be under Thread Group. Determining number of users, ramp-up and loop count is allowed by this section. I think explaining it here necessary with an example to make you clear; for example, you have 100 users and expect 10 of them launch at the same time when you start your test. Then, we can say that 100/10=10 sec is your ramp-up period time. This means first 10 users launch in the first sec, 20 users launch at the end of 2nd sec, 30 users launch at the end of 3rd sec and make your test. At the end of 10th second, your whole users will be launched and finish the test. JMeter brings “Forever” option ticked in default which signs your test will be tested with endless loop but even so, you’re able to change it. Test Run Scheduler placing 1.9 version allows you to add start and end time.

  • Controllers: They can change order and queue of your test and can be bottom element of Samplers, Config Elements or Logic Controllers. The point to be considered is that controllers can change the request itself and thus, JMeter may fall into repetition.

  • Configuration Element: It is used for configuring and changing samplers which are made a request to the server. You should put it on samplers’ top level that you want to structure. Let’s view some of frequently used of them:
  • CSV Data Set Config: It reads data from an external CSV file, writes them in variable fields and uses them sampler request.
  • User Defined Variables: It is used for creating variables to be used.
  • JDBC Connection Configuration: It is used for creating JDBC connection settings with JDBC Request sampler and also it is used for creating a database test plan.
  • Login Config Element: It is used for creating default credentials for samplers using username and password.

  • Listeners: They’re used for viewing and analysing performance test results in tabular or graphical forms. Besides, it provides users different time metrics with each other. Let’s look into the ones frequently used:
  • Aggregate Graph: It is used to view test results in both tabular (report form) and graphical form.
  • View Results in Tree: It is used to store and provide test results for each samplers. You can display the whole response body, response header, request sent to server and their bodies here.
  • View Results in Table: It is one of listeners that you can reach response header, latency, sample time etc.

How to Test Web Services with JMeter?

As usual, progressing will be developed through Test Plan. You should assign user/s to your test plan that you’ll actualise. Therefore, Thread Group to be created by you will be worked for you. The number of users and information belonging to some tests I mentioned above can be changed here. And after that step, we can start with the requirements of testing a service; for example, “where should I send my request to and from?” question should come to our mind as first. HTTP Request from Sampler that provides you to create your request details and revise in it will help you about your question. Here, you introduce Server/IP and your request will be forwarded through a specific port to a server you want to send and you’ll be able to get a response. And also you can define, match or associate values you can see in the Result section in addition to the request you have in the Parameters section. The only thing you should do is running your test plan and finding your test results in Sampler Result under View Results in Tree section.

How to Config JDBC (Java DataBase Connectivity) Connection?

I am sure that you have a test environment to store data on your web applications that you want to test and also you need to retrieve data from there and write this data on an external file. All of these works I mentioned above can be executed via JDBC configuration.

Firstly, you should specify number of Thread Group (users) and their features over test plan that you want to actualise. They are the next requirements that you should add JDBC Request from Sampler and JDBC Connection Configuration allowing you to make your settings. You should run your test plan after typing Database URL and JDBC Driver Class info into their field, your query into the request query field. As always, you can view your results from the View Results in Tree listener.

Please follow and like us:

Test Tools & Importance of Testing

DAMLA IYILIKSEVER | BUSINESS ANALYST

Which firm or person that is aiming to develop software and focuses on publishing a good product does not want it to be perfect? I seem to hear you saying: “Of course, it should be error-free! This is deadly important for my company’s prestige.”

Well… Have we ever thought that how can we be sure about is our product “perfect” or not?

People have the main idea and accept the job is all done if they are near to finish line. They miss a very important point here; Test Processes. Most of the companies do not attach importance to testing, do not write the test codes and they consider testing as a waste of time. This contrarian understanding with time management is adopted by many companies.

Let’s make you clear about what is software testing. Software testing is made for analyzing software supplies with the expected needs and features or not. Please note that it is a big mistake believing that these test phases are only for testing software. Screens’ usability (visuality, encolouring, type font and size, design, functional-none functional details) that end-users will experience can be tested from the point of end-users.

Even you have a problem with the situation of your product’s delay because of the test phase and processes, testing will decrease your costs in a long-term by including early bug determination and bug prevention actions. It provides you to gain trust and satisfy of your customers and thus, your product’s quality will be raised. The truth is that prestige will come along with these test requirements indisputably. There are various test management tools can make you satisfied with your product’s quality issue in the market. Let’s check on them:

TestRail: It is a test tool which is preferred by large informatics companies due to its ability to be accessed via web and to configure the fields it contains and to be integrated with error tracking tools. It can provide users procure many report types (created test cases, taken bugs, test comparisons, feature distribution, test plan summary, user workloads etc.). “Todos” section includes user’s and team members’ test conditions and it’s possible to filter case status and scenarios. There are various integrations with platforms like Lighthouse, Manuscript, Mantis, PivotalTracker, Rally, Redmine, TFS, Trac, Vault, VisualStudio, Online, You Track, Bitbucket, BugTracker.NET, Bugzilla, Fixx, GitHub, JIRA. I guess management screens’ convenience is one of the reasons of preference by firms. Together with its benefits and features, there are some reasons not to choose by several companies as you can guess because of the cost factor.

Case.io: Accessing it via web, usage simplicity and convenience, being a Cloud-based tool, adding “unlimited” team members, preparing result reports and sharing them through URLs, sharing all test conditions with all team members in “shared steps” section, adding milestone feature, filtration of case status and Import/Export option may make this tool attracted for the companies. Here is the bright side, you can reach these all features on free version. It has JIRA, Redmine, YouTrack, Slack integrations at the same time and the information on many other platforms’ integration will exist in near future has given. It has a paid version normally but of course, you can choose the version according to your needs. It is more acceptable that the other test management tools because of its more convenient priced feature. Thus, you can run your all test processes with it.

We have talked about free/paid test management tools you can use up to now. On the other side, there are other test types like performance, load and functional that you can actualize. Now, I want to introduce you a lifesaver: JMeter which can test dynamically produced SOAP (Simple Object Access Protocol) and REST (Representational State Transfer) based web services besides static files like HTML, image, CSS, js.

JMeter: JMeter is a %100 java language based test tool that’s why Java Runtime Environment (JRE) or Java Development Environment (JDK) should be downloaded on your device. JMeter saves the test plans as XML format and provides you use over and over again. It is open source and can be run with multi-threads. You are able to simulate user requests with it just like the real users make web requests while they’re using web applications. It uses HTTP protocol to seem as virtual user. A test plan given to JMeter determines which pages will be called when and with which information. It starts surfing your web pages by connecting to your application with HTTP protocol if the test plan is put in process in the direction of this information. Simulated test scenarios by it, web application requests and inputs can be fictionalized by differentiating just like more than one user are running the scenario at the same time.

Not finding any bugs in your product means you don’t have a bug with the current scenarios and methods you have, but you still may have bugs. It’s nearly impossible to test without any missing details. At that point, your risk factors should be counted and you should make your tests until see there is no risk according to your checks.

 

Resources:

https://jmeter.apache.org/  & JMeter Application

https://www.gurock.com/testrail & TestRail Application

https://app.qase.io/projects  & Case.io Application

 

 

 

 

 

 

Please follow and like us: