IoT Testing Complexities

Posted on July 25, 2017 by Prabhu

A comprehensive QA strategy is essential to cover the dimensions of IoT testing. The strategy should include the types of testing, test lab setup, testing tools and simulators/emulators that are to be deployed. Considering the practical hiccups in generating big data from the thing (device) in a testing environment, it is crucial to evaluate data simulation and virtualization methods.

Stubs can be considered as an option during early stages, whereas, data recorders can assist as an alternative at the later stages. Beyond test planning and data simulation, metrics-driven exhaustive test execution is performed to achieve a stable system. QA team can split IoT test area into two layers i.e. device interaction layer and user interaction layer. However, QA has to be performed across both the layers. It is actually easier to identify techniques and the types of testing that can be adapted to each layer to enhance the QA strategy.

The device interaction layer

This layer acts as the connectivity between the software and the hardware components of a real-time ‘IoT environment interact’. A typical example will be a Bluetooth device transmitting real-time data to a mobile device application. Sometimes, a lot of interaction testing will be done on the functional side of QA.

However, other types of testing could also be required. IoT testing will cover the spectrum of other required elements listed below, in addition to typical software testing:

Conformance with standards: These are mostly device performance traits that are precise to the devices and sensors. These attributes must be validated against the standards of the device and its communications protocol. Hardware vendors perform most of these tests, but there could be a certain domain or use-case specific requirements such as the use of such devices in an environment that was not tested.

Interoperability: The ability of different devices to support the required functionality among themselves, other external devices and implementations.

Security: With billions of sensors in the making, it’s crucial to tackle data privacy and the security concerns across the IoT ecosystem.

The following are the different types of security testing requirements:

• Identity and authentication
• Data protection
• Data encryption
• Storage data security in local
• Remote clouds

The user interaction layer:

This layer is the touch point between the thing (IoT Device) and the user. The success of the overall system depends on the seamless user experience.

Key testing areas in this layer include:

Network capability and device level tests: The specific aspects of network communication such as connectivity are validated by simulating different network modes in addition to device-level validation such as energy consumption tests, etc.

Usability and user experience: Usability and user experience are important in terms of the real-time usability; it involves both human and machine interaction and also the real-time experience that the IoT system provides.

For example, contactless payments compared with a physical card-based payment. 

The IoT services and back-end IoT environment:

While integration testing of the interfaces is a key, there is a complex data layer that comes into play.

For example, a classic IoT system boxes a complex analytical engine to ensure an exceptional user experience. This creates a QA environment to assist validation of such interface by addressing the growing data volume, velocity, and variety challenges. The front-end validation environment can be done by assembling data recorders and simulators. The service and data layer validations will involve complex simulation such as the generation of millions of sensor hits, machine learning algorithms and the time-boxed traffic. There are a few methods to create such an ecosystem; For example, leveraging sandboxes of development services or creating mock environments using virtualization tools. However, numerous implementation synergies are required to establish a working set of environments for through services and back-end validation platform.

Let’s discuss more IoT testing in our upcoming blogs.

Please comment on your interested topic in IoT.

A day in the life of a Game Tester!

To give an idea of what a Game tester’s day looks like, we are talking about a guy who goes by the gamer tag ‘Dalda’

Is game testing a job or a career?

While the game testing career starts as a lower paying job, there is high potential for anyone to grow in the industry very quickly. Those who have an urge, commitment and wish to make a name for themselves in gaming industry, game testing provides a big platform to grow. If one is focused long enough and keep learning and stay current, he/she can make a managerial role within a short span of 10 years.

How did you get started?

I always found myself playing a lot of video games going back as far as I can remember. From my college days, I was interested in computers plus my long standing interest in video games lead me to conclude that I wanted to be a part of the gaming industry.

The first job I got was in company, one of whose Executive was a friend’s Uncle. Knowing the bunch of us and our addiction to gaming we got offered a job.

What do you do all day?

Playing games was something I knew how to do However, learning how to be a tester, I had to unlearn what I knew, which was killing bad guys, smashing stuff and level up faster. That does not help a tester, test a game effectively. On a regular day my schedule was pretty tight. I had do to all sort of background stuff like anyone does at their job, like keeping track of what they do.

What skills do you think are most useful for a QA tester?

Attention to detail, and understanding the game. Learning the game properly and in detail will help us to understand patterns within the game, which will help to break the game and find more bugs.

What’s the pay like?

Unlike software engineers, game testing is not a highly paid job at entry. A game tester with 2 years of exposure would have more knowledge than a software tester who holds 5 years of experience. The learning curve is much steeper. All that will help one to grow quickly and get to bigger roles with better pay.

Are there any games which involve dread testing?

Definitely. Testing a game in a foreign language, without any support documentation or without anyone to explain the game, makes your job a lot harder.

Do you have a say in what games you test?

No. You go where you are asked to go and are expected to work on anything that is sent your way.

What’s the best and worst part of being a QA tester?

Best part is that you get to learn so much. You see a game when it is just in the developmental stages. You get to track the whole game through its various phases. You get to lend a hand in shaping the game and making it better for everyone to enjoy. You get to see the game through to the end. And at the end of it all, you are happy because you know you helped make the game better and everyone is enjoying the game!

Worst part would be, after working for a whole day on a game, I ended up dreaming myself as a character stuck in the game and being unable to clear the level to get out!

Do you play the games you test in your down-time?

Yes, occasionally. One of the games I tested – Section 8, my colleagues and I ended up staying late for a few hours past the end of day just to enjoy the game. Some more games in that list are CS: Condition Zero, Unreal Tournament.

Has the job affected your enjoyment of gaming?

After a small initial stage of not wanting to look at a video-game after working for a whole day on video-games, I have gone back to finding and playing new games in my free time and it is still a lot of fun.

What’s your favorite game and why?

Last of Us! No game has had me being so mentally involved with the protagonist.

Favorite movie based off a video game?

Halo Legends and Resident Evil series. These movies stayed close to the video game story line, while most of the other movie versions of the games did not portray the story line nor convey the experience when you play the game.

Learning Management System Compliance

A very common scenario that any learning management system (LMS) vendors experience is, “A potential client reaches at their exhibition booth and asks, “Is your LMS complies with regulatory standards?”

The answer is not very simple.

The worry with compliance is that it is driven by laws and regulations, which require specific training. The desire of companies is to avoid liabilities, business objectives implementation, reduction of paperwork.

To understand the topic thoroughly, we have to differentiate between the ability to track compliance in a number of areas and as a piece of software, a learning management system should be compliant with certain standards and regulations. It is crucial to understand that laws and regulations are explicit to individual countries or states, and will vary widely from one jurisdiction to another.

The below definitions and examples are drawn from the United States and Canada but there will be similar issues in the regulation of other countries as well.

Below are main compliance issues that a learning management system vendor needs to consider:

Accessibility standards – In the United States, accessibility regulations come under the Americans with Disabilities Act (ADA) and Section 508 (29 U.S.C. ‘794d) of the 1998 Rehabilitation Act. These standards follow the World Wide Web Consortium’s (W3C) Web Content Accessibility Guidelines 1.0. LMS adhering to these standards will help to increase the user base.

Regulatory compliance tracking – LMSs are basically large databases and are often questioned to track regulatory compliance with specific government regulation.

Example: In United States, Health Insurance Portability and Accountability Act (HIPAA), and the Occupational Safety and Health Administration (OSHA) regulations are included.

In Canada, the requirements of training for the Workplace Hazardous Materials Information System (WHIMIS) are at times tracked by an LMS.

Security standards – Many organizations require secured systems to store personal information. This is true in the medical field but can apply in other areas as well.

These requirements will include

• maintenance of audit trails,
• deployment of completely closed systems,
• usage of digital signatures, such as the US Federal Drug Administration regulation FDA 21 CFR Part 11 for the medical, biotech, and pharmaceutical industries,
• use of high degrees of encryption for groups such as the banking industry.

Interoperability standards – One of the most basic demands of LMS users is the interoperability of courses developed by different developers in the same LMS. It led to the development of one of the first set of LMS standards like those of the Aircraft Industry CBT Committee (AICC). In 1997, the IMS Global Learning Consortium (IMS GLC), a non-profit organization keen in setting stipulations and standards for the learning industry, was formed. It issued many sets of specifications since that date. In the same year the announcement of the Advanced Distributed Learning (ADL) initiative of the US Department of Defence. ADL developed the Shareable Content Object Reference Model (SCORM) and the ADL Registry of SCORM compliant software. LMS vendors are well aware of both AICC and SCORM, and generally are compliant with both standards.

However, most LMS implementations don’t work well with these standards and often require a period of adjustment and tuning to make them work seamlessly.

Tracking training for certification – There are many regulatory compliances for training in specific industries, where employees are required to be certified before being employed and require to be recertified on a regular basis. Many LMSs track certification and will trigger automatic alerts as the date for recertification approaches. This need can be driven by legislation or by standards imposed by a specific industry or company.

Tracking training for liability reduction – Training can aid reduce the liability for employers in controversial areas. Courses or educational materials on such things, such as sexual harassment, or employment discrimination; might be much less expensive to provide than a settling of lawsuits in case of disputes.

Reduction in paperwork – In conclusion, a fully functional LMS will have features that reduce workload and paperwork for compliance management. Such features include auto-enrolment in compliance training based on job, automated alerts to managers and workers on failure to complete compliance training, assessment and evaluation of the training, certificate issue on training completion, and many other configurations for tracking and reporting in this area.

So the answers to questions on compliance and learning management systems are not simple and are multidimensional.

Hopefully, this article will help you to sort out what you need in this area.

The Basics of Mobile Web Apps Testing On Real Devices Using Selenium / Appium

First Thing First: Appium Equals Selenium

“Appium is just like Selenium – but only for mobile apps and games “. One should have heard about it many times, but in fact, Appium is much more than that.

It is also suitable for mobile web apps testing, where real devices and real browsers are used in testing. With the same effort and cost, a test automation framework can be built with Appium using real stuff or use an emulator/simulator for the automation.

In a nutshell, Appium is a mobile test automation tool that works for native, hybrid as well as mobile web apps for iOS and Android with the aid of Selenium server. Appium is an excellent choice for test automation framework as it can be used for all these different app/web types.

Fundamentally, Appium derives its root from Selenium and it uses JSON Wire Protocol inside to interact with iOS and Android apps using Selenium’s WebDriver.

In its architecture, Appium is an HTTP server written in Node.js which creates and handles multiple WebDriver sessions. Appium starts tests on the device and takes notes for commands from the main Appium server. It is basically the same as the Selenium server that gets HTTP requests from Selenium client libraries.

Appium has proven its flexibility and scalability in a native mobile app and game testing, as those apps and games tend to be much identical on both platforms i.e. Android and iOS. The benefit of having this cross-platform testing framework helps by using the identical script for running app/game on either platform. The same applies to the mobile web.

Appium scripts run smoothly when web testing is done on Chrome, Firefox, Safari, or WebKit-based browser on Android and iOS where applicable.

Another significant benefit of Appium is that users can write tests using their favourite development tools, environment and programming language, such as Java, Objective-C, JavaScript, PHP, Ruby, Python or C#, among many others.

How to get started and what things to consider?

Don’t worry about the learning curve. If you are familiar with Selenium, then you’ve got Appium covered: They use WebDriver and DesiredCapabilities in the same way. There is a great chance the existing Selenium scripts work as they are with Appium setup. However, configuring an application to run on Appium has a lot of similarities to Selenium — for example, as those of DesiredCapabilities.

Don’t know about Selenium – Follow the URL http://www.seleniumhq.org/ 

The below instructions provide everything to ensure a smooth start with Appium, real devices and real web browsers.

Bear in mind that the mobile ecosystem is very different compared to the desktop world. As mobile OEMs are building their own stuff, they also tend to ‘differentiate’ on various standards – such as browsers.

In fact, mobile browsers – regardless how standard they are – is also one of the areas where almost all devices manufacturers want to add their ‘innovation’.

This is quickly leading to a happy mix of different web kit versions with different functionalities; when combined with the OS version, hardware configuration, sizes of displays, memory and so on can significantly ‘differentiate’ what user gets as a form of these browsers. Building your test scripts with Appium/Selenium and automated testing can quickly get the understanding that how well does the web work on those devices and browsers.

Another area that any mobile web developer needs to pay attention at is performance:

Web page rendering time varies expressively from one device to another.

To ensure satisfactory end user experience it should be tested that how the CSS & Javascript renders across devices. There are already great benchmarks – such as BrowserMark by Rightware can be used to measure the performance of the hardware together with certain versions of browsers. This gives an understanding of the workload with certain device and web browser combination.

Selenium has already cemented its place in web test automation standard and Appium is providing very clean high-level API that can be quickly adopted by Selenium developers. It nicely abstracts a lot of the messy stuff related to parallel running tests.

Testing a CRM Application

CRM systems are progressively becoming strategic assets for organizations, helping them manage the entire customer life-cycle, streamline operations and improve customer satisfaction.

According to Gartner, CRM implementation failure rates exceed 50% due to lack of effective CRM strategy and user adoption.

Ensuring a successful CRM implementation requires focus on business objective, robust business processes and a focus on the Quality of implementation.

An independent Quality Assurance and Testing process helps identify real challenges for CRM implementation, assess robustness and ensure adherence to user requirements. This results in higher success rate. Having a verification and validation Testing Team helps achieve greater quality in terms of both technical & business specifications and higher ROI.

A buggy or improperly implemented feature of a CRM application can have direct impact on customer relationship and hurt the revenue. Equally, a well-implemented CRM can help you gain customer insight and improve customer satisfaction and loyalty.

CRMs also contain a lot of sensitive data about company as well as customers. Any CRM testing program should ensure that the data are accurate and secure during the storage and retrieval transactions.

A typical CRM work flow will look like the below demonstrated image.

Testing Cycles for a CRM Application

Data Integrity and Conversion Check

The first test cycle in CRM testing should focus on issues related to data quality and conversion. At every step of a migration or update the testers should verify that the CRM is working as expected for both with and without data.

Below is the check list to ensure data quality:

1. Duplication of data: No redundancy should be allowed.
2. Hidden data is indeed hidden: Only the appropriate data should be visible according to the user roles and type.
3. Data maps correctly: Grid issues where a selected row’s data goes out of alignment with the fields when scrolled down. This should be eliminated.
4. New and updated data should be saved properly: Customer information and card information should save and update properly.
5. Partial and full search work as desired: Users should be able to search by first name, last name, company, card is deemed necessary.
6. No data is missing: All of the required data should be available to the right user levels.
7. Graphs should represent data correctly: Critical data like store-specific and program-specific filters and sales percentages should be accurate.
8. Data sorting: All of the fields that need to be sortable should works as desired.
9. CRM installs correctly: No critical GUI issues should arise during installation even when the user misses the data during installation.
10. Data saved in one field does not move to another field after saving it: Fields like address, which may need more than one field, should save to the correct fields.
11. Check that data which is not supposed to be editable is indeed not editable: Data like date and time of transaction should not be editable.

Functional Testing

This deals with the application’s function and ensures that it fits during the migration of data from another legacy system. The values and other data should be populated in the correct fields.

Below is functional checklist for CRM Testing –

• Organization wide default
• Access level
• Accessible to hidden data
• Transaction processes
• Connection lost
• Proper exception handling
• Avoid data mismatch
• Pre-Authorization
• Post-Authorization

Performance Testing

The CRM application should be tested under pressure or stress or load. This can measure the effects of concurrent users on the system’s performance.

Performance Check List for CRM Testing –

• Response time
• Server utilization
• Peak hours testing
• Loading speed
• Error with any other integrated applications due to this
• Maximum number of users

Integration Testing

This will be a challenge in CRM applications where we will be migrating or integrating third party applications into the CRM.

Listed are the must see features during integration testing of a CRM

• Field and Labels matching
• Dependencies Integration
• Date format
• Reports
• Data Validation
• Sorting order

Regression Testing

There will be regression testing always in an application like CRM where there is a need to test before migration and after migration. After migration whether the values are populated into the correct fields and validations are matched with the data. Other fields should not be disturbed because of this. This will measure Functional correctness and completeness of the CRM application. With the help of regression testing the reliability and performance of the application can be ensured.

Security Testing

CRM involves high volume of confidential data which should be secured. A proper security test plan allows how well the system protects the data against unauthorized internal and external access.

Are you into CRM Testing?

WannaCry – Taking Ransomware Protection to Next Level

What is WannaCry?

A worldwide cyber-attack has been happening since Friday, affecting more than 200,000+ organizations in 150+ countries. The WANNACRY RANSOMWARE ATTACK has rapidly become the nastiest digital disaster to strike the internet, crippling transportation and hospitals globally. But, it progressively appears that it is not the work of hacker brains. Instead, cyber-security detectives see the recent breakdown of cyber-criminal scheme, which reveals amateur mistakes made at every steps.

How does Ransomware Work?

Ransomware is a kind of cyber-attack, in which, hackers take control of a computer system and block access to it until a ransom is paid. The cyber criminals need to download a type of malicious software onto a system within the network to gain access to the system. This is often done by making the victim click on a link or download it by mistake. Once the software gets into the victim’s computer, the hackers can launch an attack that will lock all the files it finds within the network. It tends to be a gradual process with files being encrypted one after the other.

Though the infection stage is somewhat different for each Ransomware version, the key stages are as follows:

• Initially, the victim gets an email that includes a malicious link or a malware attachment. Alternatively, the contagion can originate from a malicious website that delivers a security exploit to create a backdoor on the victim’s PC by using a susceptible software from the system.
• If the victim clicks on the link or downloads and opens the attachment, a downloader (payload) will be placed on the affected PC.
• The downloader uses a list of domains or C&C servers to download the Ransomware program on the system.
• The contacted C&C server responds by sending back the requested data.
• The malware then encrypts the entire hard disk content, personal files, and sensitive information. Everything, including data stored in cloud accounts (Google Drive, Dropbox) synced on the PC is encrypted by the malware. It can also encrypt data on other computers connected to the local network.
• Then a warning pops up on the screen with instructions on how to pay for the decryption key.

Taking Ransomware Protection to Next Level

One should take the threat of Ransomware seriously and do something about it before it smashes the data.

Here are few precautions to

Local PC:

Step 1: Do not store important data only on your PC. Take 2 backups of data: on an external hard drive and in the cloud – Dropbox/Google Drive/etc.

Step 2: The Dropbox/Google Drive/OneDrive/ applications should not be turned on by default.

Step 3: Turn off macros in the Microsoft Office suite – Word, Excel, PowerPoint, etc.

In the browser:

• Block and set the plugins to ask for permissions for the following plugins from the browser: Adobe Flash, Adobe Reader, Java and Silverlight and activate the plugins when needed.
• Adjust the browsers’ security and privacy settings for increased protection.
• Update all outdated plugins and add-ons from my browsers.
• Use an ad-blocker to avoid the threat of potentially malicious ads.

Online Behaviour:

• Never open spam emails or emails from unknown senders.
• Never download attachments from spam emails or suspicious emails.
• Never click links in spam emails or suspicious emails.
• Use Anti-ransomware security tools such as Norton, Bitdefender, Kaspersky, Trend Micro Internet Security, Zemana Anti-malware.
• Use a reliable, paid antivirus product that includes an automatic update module and a real-time scanner.

“Should I pay the ransom or not?”

The answer is a big NO.

Paying the ransom does not give guarantee that the online criminals at the other end of the Bitcoin transfer will give the decryption key. And even if they do, there will be further greedy attacks, which will become a never-ending malicious cycle of cyber-crime.

Putting things into perspective, 1 out of every 4 cyber-crime victim who paid the ransom didn’t get their data back. They lost both the information and their money.

How to get the data back without paying the ransom?

There are many versions and types of Ransomware, but cyber security researchers are working round the clock to break the encryption that at least some of them use. There are many other cryptoware strains that are well coded and only specialists are able to crack.

Not sure if your system in secure? Don’t Worry!!

Our Security Testing experts will guide You

To recover the data without funding Ransomware creators, we have put together a significant list of Ransomware decryption tools which can be used.

• OpenToYou
• Globe3 decryption tool
• Dharma Decryptor
• CryptON
• Alcatraz
• HiddenTear
• NoobCrypt
• CryptoMix/CryptoShield decryptor
• Damage ransomware
• .777 ransomware
• 7even-HONE$T
• .8lock8 ransomware decrypting tool
• 7ev3n decrypting tool
• Agent.iih
• Alma decrypting tool
• Al-Namrood
• Alpha
• AlphaLocker
• Apocalypse
• ApocalypseVM
• Aura
• AutoIt
• Autolocky
• Badblock
• Bart decrypting tool
• BitCryptor
• BitStak
• Chimera
• CoinVault
• Cryaki
• Crybola
• CrypBoss
• Crypren

• Crypt38
• Crypt888
• CryptInfinite
• CryptoDefense
• CryptoHost
• Cryptokluchen
• CryptoTorLocker
• CryptXXX
• CrySIS decrypting tool
• CTB-Locker Web
• CuteRansomware
• DeCrypt Protect
• Democry decrypting tool
• DMA Locker decrypting tool + DMA2 Locker decoding tool
• Fabiansomware
• FenixLocker
• Fury decrypting tool
• GhostCrypt decrypting tool
• Globe / Purge
• Gomasom
• Harasom
• HydraCrypt
• Jigsaw/CryptoHit
• KeRanger
• KeyBTC
• KimcilWare
• Lamer decrypting tool
• LeChiffre
• Legion
• Linux.Encoder
• Lock Screen ransomware
• Locker
• Lortok

• MarsJoke
• Manamecrypt
• Mircop decrypting tool + alternative
• Merry Christmas / MRCR decryptor
• Nanolocker
• Nemucod
• NMoreira ransomware
• ODCODC
• Operation Global III Ransomware
• Ozozalocker ranomware decryptor
• PClock
• Petya
• Philadelphia
• PizzaCrypts
• Pletor
• Pompous
• PowerWare / PoshCoder
• Radamant
• Rakhni
• Rannoh
• Rector
• Rotor
• Scraper
• Shade / Troldesh
• SNSLocker
• Stampado
• SZFlocker
• TeleCrypt
• TeslaCrypt
• TorrentLocker
• Umbrecrypt

Please read about how these tools work before using it as a solution.

Do keep in mind that decryptors could become outdated due to constant updates and new versions released by cyber criminals. This is a never-ending battle, which is why we should focus on prevention and having multiple backups for your data.

API Testing of Online Communication App

http://www.indiumsoft.com/casestudy/integrated-api-testing.pdf

Successfully implemented QA Methodology and process