Many Companies don’t have resources or can’t afford to hire the
required number of testers on the project. So what could be the solution
in this case?
The answer is simple. Companies will prefer to have skilled testers instead of a army of testers!
So how can build skilled testers on any project?
You can improve testers performance by assigning him/her to the single project.
Due to this the tester will get the detail knowledge of the project
domain, Can concentrate well on that project, can do the R&D work
during the early development phase of the project.
This not only build his/her functional testing knowledge but also the project Domain knowledge.
Company can use following methods to Improve the Testers performance:
1) Assign one tester to one project
for long duration or to the entire project. Doing this will build
testers domain knowledge, He/She can write better test cases, Can cover
most of the test cases, and eventually can find the problem faster.
2) Most of the testers can do the functional
testing, BV analysis but they may not know how to measure test
coverage,How to test a complete application, How to perform load
testing. Company can provide the training to their employees in those areas.
3) Involve them in all the project meetings, discussions, project design so that they can understand the project well and can write the test cases well.
4) Encourage them to do the extra activities other
than the regular testing activities. Such activities can include Inter
team talk on their project experience, Different exploratory talks on
project topics.
Most important is to give them freedom to think outside the box so that they can take better decision on Testing activities like test plan, test execution, test coverage.
If you have a better idea to boost the testers performance don’t forget to comment on!
Thursday 6 September 2012
FIND BUGS IN APPLICATIONS
A very good and important point. Right? If you are a software
tester or a QA engineer then you must be thinking every minute to find a
bug in an application. And you should be!
I think finding a blocker bug like any system crash is often rewarding! No I don’t think like that. You should try to find out the bugs that are most difficult to find and those always misleads users.
Finding such a subtle bugs is most challenging work and it gives you satisfaction of your work.
Also it should be rewarded by seniors. I will share my experience of
one such subtle bug that was not only difficult to catch but was
difficult to reproduce also.
I was testing one module from my search engine project. I do most of the
activities of this project manually as it is a bit complex to automate.
That module consist of traffic and revenue stats of different
affiliates and advertisers. So testing such a reports is always a
difficult task. When I tested this report it was showing the data
accurately processed for some time but when tried to test again after
some time it was showing misleading results. It was strange and
confusing to see the results.
There was a cron (cron is a automated script that runs after
specified time or condition) to process the log files and update the
database. Such multiple crons are running on log files and DB to
synchronize the total data. There were two crons running on one table
with some time intervals. There was a column in table that was getting
overwritten by other cron making some data inconsistency. It took us
long time to figure out the problem due to the vast DB processes and
different crons.
My point is try to find out the hidden bugs in the system
that might occur for special conditions and causes strong impact on the
system. You can find such a bugs with some tips and tricks.
So what are those tips:
1) Understand the whole application or module in depth before starting the testing.
2) Prepare good test cases before start to testing. I mean give stress on the functional test cases which includes major risk of the application.
3) Create a sufficient test data
before tests, this data set include the test case conditions and also
the database records if you are going to test DB related application.
4) Perform repeated tests with different test environment.
5) Try to find out the result pattern and then compare your results with those patterns.
6) When you think that you have completed most of the test conditions and when you think you are tired somewhat then do some monkey testing.
7) Use your previous test data pattern to analyse the current set of tests.
8) Try some standard test cases for
which you found the bugs in some different application. Like if you are
testing input text box try inserting some html tags as the inputs and
see the output on display page.
9) Last and the best trick is try very hard to find the bug 
As if you are testing only to break the application!
As if you are testing only to break the application!
Meantime you can comment out more tips here.
Sunday 2 September 2012
TESTING WEEKLY STATUS REPORT
Writing effective status report is as important as the actual work you did! How to write a effective status report of your weekly work at the end of each week?
Weekly report is important to track the important project issues, accomplishments of the projects, pending work and milestone analysis. Even using these reports you can track the team performance to some extent. From this report prepare future actionables items according to the priorities and make the list of next weeks actionable.
So how to write weekly status report?
Follow the below template:
Prepared By:
Project:
Date of preparation:
Status:
Prepared By:
Project:
Date of preparation:
Status:
A) Issues:
Issues holding the QA team from delivering on schedule:
Project:
Issue description:
Possible solution:
Issue resolution date:
Project:
Issue description:
Possible solution:
Issue resolution date:
You can mark these issues in red colour. These are the issues that requires managements help in resolving them.
Issues that management should be aware:
These are the issues that not hold the QA team from delivering on
time but management should be aware of them. Mark these issues in Yellow colour. You can use above same template to report them.
Project accomplishments:
Mark them in Green colour. Use below template.
Project:
Accomplishment:
Accomplishment date:
Project:
Accomplishment:
Accomplishment date:
B) Next week Priorities:
Actionable items next week list them in two categories:
1) Pending deliverables: Mark them in blue colour: These are previous weeks deliverables which should get released as soon as possible in this week.
Project:
Work update:
Scheduled date:
Reason for extending:
Project:
Work update:
Scheduled date:
Reason for extending:
2) New tasks:
List all next weeks new task here. You can use black colour for this.
Project:
Scheduled Task:
Date of release:
List all next weeks new task here. You can use black colour for this.
Project:
Scheduled Task:
Date of release:
C) Defect status:
Active defects:
List all active defects here with Reporter, Module, Severity, priority, assigned to.
List all active defects here with Reporter, Module, Severity, priority, assigned to.
Closed Defects:
List all closed defects with Reporter, Module, Severity, priority, assigned to.
List all closed defects with Reporter, Module, Severity, priority, assigned to.
Test cases:
List total number of test cases wrote, test cases passed, test cases failed, test cases to be executed.
List total number of test cases wrote, test cases passed, test cases failed, test cases to be executed.
This template should give you the overall idea of the status report.
Don’t ignore the status report. Even if your managers are not forcing
you to write these reports they are most important for your work
assessment in future.
Try to follow report writing routine. Use this template or at
least try to report it in your own words about the overall work of
which you can keep some track.
Do you have any better idea for this routine work? Comment it out!
TIPS TO HANDLE ANY JOB INTERVIEW SUCCESSFULLY
Interviews have always been a nerve racking experience. A situation
where you are judged on your performance for a job. Everybody gets the
jitters when it comes to interviews. Relax! Don’t panic. You need to overcome the nervousness.
Job Interview Tips and advice Applicable for Any Job Seeker Looking for a Dream Job.
No matter which career path you want to choose below are the best tips to help you land your dream job.
1. Always do your homework well before walking into an interview. Make sure you have complete knowledge about the company and the role.
2. Know yourself. Remember first impression is the last impression. Demonstrate your capabilities and qualities and how well you can serve them. Don’t be overconfident and aggressive.
3. You should know your competency and transferable skills. Competency skills are the skills matching your job profile and transferable skills which you acquired through other jobs, personal activities.
4. Social networking sites like Facebook, Orkut, Linkedin can be used for work opportunities and conversing with other people improving your interpersonal skills.
5. Be clear about what you want to achieve in life and about your career objective. It will keep you focused. You don’t have to do anything for the heck of it.
6. Your CV is vital for a successful interview. Never bluff, include all your skills and experience to give you a competitive edge.
7. Prepare well for an interview. You can make notes of interview questions which are most likely to be asked. Practice your answers. This will boost your confidence.
8. Work on your communication skills. Remember having a good technical knowledge without effective interpersonal skills will not take you anywhere. Be expressive and a good conversationalist. Dazzle the interviewers with eloquent speech.
9. Make sure you can support your strengths by giving examples. You can prepare before but don’t falter while talking. It will not create a good impression.
10. When asked about your weaknesses acknowledge them. If you are not able to describe, it signifies that you lack self awareness. You can’t be perfect in everything.
11. Always be presentable while dressing for the interview. Your attire should be according to the role, culture and yourself. Please no tacky and brash clothing and accessories. You don’t need to be glammed up.
12. Spend time on personal grooming. This will keep you calm. You don’t have to present yourself as a person full of nervous energy and fidgets.
13. On the D day, relax. Be comfortable and wear a smile. And Voila! You will definitely crack the interview.
14. Your body language is very important. Your facial expressions, hand movements, posture, voice and pace should send the same message.
15. Don’t forget to make an eye contact. Your voice should be enthusiastic and do not stammer. Lack of enthusiasm will put off the interviewers.
16. Keep all your documents well organized in a folder. Also be on time, preferably 15 minutes before so you get time to settle down and calm your nerves.
17. Interview manners are very important. Bad manners will definitely be a turn off. Don’t bang the door, shake handily firmly, ask if you can take a seat, sit up straight and do not slouch.
18. When asked about remuneration. You don’t have to be blunt. Instead you can say that you expect a fair raise in terms of qualifications and experience proportionate with peers.
Now that you have some good interview tips. Be confident, gear up and don’t let yourself down.
Remember this is not the end of life if you don’t get through to the process. It’s just an interview. Good Luck!
Job Interview Tips and advice Applicable for Any Job Seeker Looking for a Dream Job.
No matter which career path you want to choose below are the best tips to help you land your dream job.
1. Always do your homework well before walking into an interview. Make sure you have complete knowledge about the company and the role.
2. Know yourself. Remember first impression is the last impression. Demonstrate your capabilities and qualities and how well you can serve them. Don’t be overconfident and aggressive.
3. You should know your competency and transferable skills. Competency skills are the skills matching your job profile and transferable skills which you acquired through other jobs, personal activities.
4. Social networking sites like Facebook, Orkut, Linkedin can be used for work opportunities and conversing with other people improving your interpersonal skills.
5. Be clear about what you want to achieve in life and about your career objective. It will keep you focused. You don’t have to do anything for the heck of it.
6. Your CV is vital for a successful interview. Never bluff, include all your skills and experience to give you a competitive edge.
7. Prepare well for an interview. You can make notes of interview questions which are most likely to be asked. Practice your answers. This will boost your confidence.
8. Work on your communication skills. Remember having a good technical knowledge without effective interpersonal skills will not take you anywhere. Be expressive and a good conversationalist. Dazzle the interviewers with eloquent speech.
9. Make sure you can support your strengths by giving examples. You can prepare before but don’t falter while talking. It will not create a good impression.
10. When asked about your weaknesses acknowledge them. If you are not able to describe, it signifies that you lack self awareness. You can’t be perfect in everything.
11. Always be presentable while dressing for the interview. Your attire should be according to the role, culture and yourself. Please no tacky and brash clothing and accessories. You don’t need to be glammed up.
12. Spend time on personal grooming. This will keep you calm. You don’t have to present yourself as a person full of nervous energy and fidgets.
13. On the D day, relax. Be comfortable and wear a smile. And Voila! You will definitely crack the interview.
14. Your body language is very important. Your facial expressions, hand movements, posture, voice and pace should send the same message.
15. Don’t forget to make an eye contact. Your voice should be enthusiastic and do not stammer. Lack of enthusiasm will put off the interviewers.
16. Keep all your documents well organized in a folder. Also be on time, preferably 15 minutes before so you get time to settle down and calm your nerves.
17. Interview manners are very important. Bad manners will definitely be a turn off. Don’t bang the door, shake handily firmly, ask if you can take a seat, sit up straight and do not slouch.
18. When asked about remuneration. You don’t have to be blunt. Instead you can say that you expect a fair raise in terms of qualifications and experience proportionate with peers.
Now that you have some good interview tips. Be confident, gear up and don’t let yourself down.
Remember this is not the end of life if you don’t get through to the process. It’s just an interview. Good Luck!
Saturday 1 September 2012
TRICKY QUESTION
Define the following along with examples
a. Boundary Value testing
b. Equivalence testing
c. Error Guessing
d. Desk checking
e. Control Flow analysis
Answer:
a. Boundary Value testing
b. Equivalence testing
c. Error Guessing
d. Desk checking
e. Control Flow analysis
Answer:
a) Boundary value Analysis: -
A process of selecting test cases/data by identifying the boundaries that separate valid and invalid conditions. Tests are constructed to test the inside and outside edges of these boundaries, in addition to the actual boundary points. or A selection technique in which test data are chosen to lie along “boundaries” of the input domain [or output range] classes, data structures, procedure parameters, etc. Choices often include maximum, minimum, and trivial values or parameters.
E.g. – Input data 1 to 10 (boundary value) Test input data 0, 1, 2 to 9, 10, 11
A process of selecting test cases/data by identifying the boundaries that separate valid and invalid conditions. Tests are constructed to test the inside and outside edges of these boundaries, in addition to the actual boundary points. or A selection technique in which test data are chosen to lie along “boundaries” of the input domain [or output range] classes, data structures, procedure parameters, etc. Choices often include maximum, minimum, and trivial values or parameters.
E.g. – Input data 1 to 10 (boundary value) Test input data 0, 1, 2 to 9, 10, 11
b) Equivalence testing: -
The input domain of the system is partitioned into classes of representative values, so that the no of test cases can be limited to one-per-class, which represents the minimum no. of test cases that must be executed.
E.g.- valid data range: 1-10 Test set:-2; 5; 14
The input domain of the system is partitioned into classes of representative values, so that the no of test cases can be limited to one-per-class, which represents the minimum no. of test cases that must be executed.
E.g.- valid data range: 1-10 Test set:-2; 5; 14
c) Error guessing: -
Test data selection technique. The selection criterion is to pick values that seem likely to cause errors Error guessing is based mostly upon experience, with some assistance from other techniques such as boundary value analysis. Based on experience, the test designer guesses the types of errors that could occur in a particular type of software and designs test cases to uncover them.
Test data selection technique. The selection criterion is to pick values that seem likely to cause errors Error guessing is based mostly upon experience, with some assistance from other techniques such as boundary value analysis. Based on experience, the test designer guesses the types of errors that could occur in a particular type of software and designs test cases to uncover them.
E.g. – For example, if any type of resource is allocated dynamically, a good place to look for errors is in the de-allocation of resources. Are all resources correctly deallocated, or are some lost as the software executes?
d) Desk checking: -
Desk checking is conducted by the developer of the system or program. The process involves reviewing the complete product to ensure that it is structurally sound and that the standards and requirements have been met. This is the most traditional means for analyzing a system or program.
Desk checking is conducted by the developer of the system or program. The process involves reviewing the complete product to ensure that it is structurally sound and that the standards and requirements have been met. This is the most traditional means for analyzing a system or program.
e) Control Flow Analysis: -
It is based upon graphical representation of the program process. In control flow analysis; the program graphs has nodes which represent a statement or segment possibly ending in an unresolved branch. The graph illustrates the flow of program control from one segment to another as illustrated through branches .the objective of control flow analysis is to determine the potential problems in logic branches that might result in a loop condition or improper processing .
It is based upon graphical representation of the program process. In control flow analysis; the program graphs has nodes which represent a statement or segment possibly ending in an unresolved branch. The graph illustrates the flow of program control from one segment to another as illustrated through branches .the objective of control flow analysis is to determine the potential problems in logic branches that might result in a loop condition or improper processing .
INCREMENTAL INTEGRATION AND TESTING
When you are testing a system, you should always adopt an incremental
approach where you gradually integrate components from different teams
and providers. Sometimes, you develop the overall structure of the
system and then add components to it. This is called top-down
integration. Alternatively, you may first integrate infrastructure
components that provide common services, such as network and database
access, then add the functional components. This is bottom-up
integration. In practice, for many systems, the integration strategy is a
mixture of these, with both infrastructure components and functional
components added in increments. In both top-down and bottom-up
integration, you usually have to develop additional code to simulate
other components and allow the system to execute.
You use an incremental approach to integration to make it easier to
discover interaction errors that occur. There are complex interactions
between system components and, when an anomalous output is discovered,
you may find it hard to identify where the error occurred. Initially,
you should integrate a minimal system configuration and test this
system. You then add components to this minimal configuration and test
after each added increment.
In the example shown in below figure, A, B, C and D are components and T1
to T5 are related sets of tests of the features incorporated in the
system. T1, T2 and T3 are first run on a system composed of component A
and component B (the minimal system). If these reveal defects, they are
corrected. Component C is integrated and T1, T2 and T3 are repeated to
ensure that there have not been unexpected interactions with A and B. If
problems arise in these tests, this probably means that they are due to
interactions with the new component. The source of the problem is
localised, thus simplifying defect location and repair. Test set T4 is
also run on the system. Finally, component D is integrated and tested
using existing and new tests (T5).
UNIT TESTING
Unit testing deals with testing a unit
as a whole. This would test the interaction of many functions but
confine the test within one unit. The exact scope of a unit is left to
interpretation. Supporting test code, sometimes called
scaffolding,
may be necessary to support an individual test. This type of testing is driven by the architecture and implementation teams.
This focus is also called black-box testing because only the details of the interface are visible to the test. Limits
that are global to a unit are tested here.
In the construction industry,
scaffolding
is a temporary, easy to assemble and disassemble, frame placed around a
building to facilitate the construction of the building. The
construction workers first build the scaffolding and then the building.
Later the scaffolding is removed, exposing the completed building.
Similarly, in software testing, one particular test may need some
supporting software. This software establishes an environment around the
test. Only when this environment is established can a correct
evaluation of the test take place. The scaffolding software
may establish state and values for data structures as well as providing
dummy external functions for the test. Different
scaffolding software may be needed from one test to another test.
Scaffolding software rarely is considered part of the system.
Sometimes the scaffolding software becomes larger than the system
software being tested. Usually the scaffolding software is not of the
same quality as the system software and frequently is quite fragile. A
small change in the test may lead to much larger changes in the
scaffolding.
Internal and unit testing can be automated with the help of coverage tools.
A coverage tool analyzes the source code and generates a test that will
execute every alternative thread of execution. It is still up to the
programmer to combine these test into meaningful cases to validate
the result of each thread of execution. Typically, the coverage tool
is used in a slightly different way. First the coverage tool is used to
augment the source by placing informational prints after
each line of code. Then the testing suite is executed generating
an audit trail. This audit trail is analyzed and reports the percent of
the total system code executed during the test suite. If the coverage is
high and the untested source lines are of low impact to the
system's overall quality, then no more additional tests are required.
Friday 31 August 2012
WEB APPLICATION LOAD, STRESS AND PERFORMANCE TESTING USING WAPT
Why most of the manual testers fail when testing websites for performance? There are couple of reasons.
- They don’t have proper tools to test website for performance and
- They don’t have required skills for performance testing.
Does that mean you should wait till your stakeholder report the performance glitches in web application you developed? Definitely not. Many testers are good at testing websites manually and they report almost every defect while testing under standard tests. BUT, when same tester performs load or stress tests they stuck either at resource (required tools) or skill level.
I suggest not to take any risk if you are committed to defect free service. Ask for required tools and train your staff for necessary skills. Today, I’m going to review load, stress and performance testing tool for websites. The tool is called WAPT – Web Application Load, Stress and Performance Testing – a cost effective and easy to learn web load testing tool.
WAPT allows you to perform website load and performance testing by creating heavy load from a single or multiple workstations. When you set and run your tests with this tool within a matter of minutes you can get performance report of your website or web application. WAPT uses powerful virtual users same as the real world users with full control over how to customize these virtual users.
- How many users can work simultaneously on your website with acceptable quality of service?
- How many visitors your website can handle by day or hour?
- What is your website response time under load?
These all questions are nothing but the measure of website “performance characteristic”.
WAPT – website performance tool performs test by emulating activity of many virtual users. Each virtual user can have its own profile settings. You can have thousands of virtual users acting simultaneously on your website performing any activity like reading or writing with your web server. Once you set number of virtual users to act on your website you have option to run your tests for specified time or specified user sessions.
Analyzing the test report:
Test result consists of charts updated in real time which you can monitor when your tests are running. The final comprehensive report is provided at the end of the tests.
Here are the important parameters to be monitored on the test report:
Error Rate: Failure rate against total number of tests run. The error may be due to the high load on server or due to the network problems and timeouts.
Response Time: Obviously a great parameter to check when you run tests for website performance. This response time indicates time required by server to provide correct reply to the request.
Number of pages per second: Number of page requests successfully completed by server per second.
How to conclude performance tests?
These performance criteria change during each test-pass with different load conditions. You need to conclude what is your acceptable load limit and whether your server is able to serve this load.
E.g.: If you expect your server to handle 100 requests successfully per second then anything below this will be failure of your server which needs to be tackled.
Testing with WAPT in simple 5 steps:
Record->Configure->Verify->Execute->Analyze
WAPT uses inline Microsoft internet explorer which is used to record your interaction with website. When you record your test all dynamic parameters are recorded as static values which can be configured later while test execution. You then need to configure each user with different settings like unique sessions, number of virtual users, values for dynamic parameters etc. Once you done with recording and configuration just verify your test if it’s ready to run and then execute performance tests if everything looks ok. Finally analyze reports to decide website performance test as accepted or failed against your set of defined standards. That’s it.
WAPT is available in two versions
- Standard version (Latest WAPT 7.5)
- Professional version of this stress and performance testing tool (Latest WAPT Pro 2.5)
What WAPT Pro can do for you?
- Use several computers to generate load on website
- Measure web server performance in terms of CPU, RAM or network usage
- You can include the execution of a JavaScript code into virtual user profiles.
If you don’t want to specify every parameter manually you can use some technology specific modules to significantly improve your test experience.
Following additional modules can be downloaded and installed along with standard or professional version of WAPT:
- Module for ASP.NET testing
- Module for Adobe Flash testing
- Module for JSON format
Finally, any review can’t be complete without the list of Pros and cons.
- Easy to use with very short learning curve
- You get run-time reports so that you can decide whether to continue the test or not, saving your big time.
- Detailed test report with graphical representation.
- Supports secure HTTPS protocol.
- 30 days free trial available!
- No scripting ability
- It’s not free
How to try this tool?
You can download 30 day trial version of WAPT from here.
That being said WAPT makes website load, stress and performance testing super easy.
Ask your queries related to WAPT tool or performance testing in comments below.
- They don’t have proper tools to test website for performance and
- They don’t have required skills for performance testing.
Does that mean you should wait till your stakeholder report the performance glitches in web application you developed? Definitely not. Many testers are good at testing websites manually and they report almost every defect while testing under standard tests. BUT, when same tester performs load or stress tests they stuck either at resource (required tools) or skill level.
I suggest not to take any risk if you are committed to defect free service. Ask for required tools and train your staff for necessary skills. Today, I’m going to review load, stress and performance testing tool for websites. The tool is called WAPT – Web Application Load, Stress and Performance Testing – a cost effective and easy to learn web load testing tool.
WAPT allows you to perform website load and performance testing by creating heavy load from a single or multiple workstations. When you set and run your tests with this tool within a matter of minutes you can get performance report of your website or web application. WAPT uses powerful virtual users same as the real world users with full control over how to customize these virtual users.
Measuring website performance:
Did you ever wonder?- How many users can work simultaneously on your website with acceptable quality of service?
- How many visitors your website can handle by day or hour?
- What is your website response time under load?
These all questions are nothing but the measure of website “performance characteristic”.
Getting Started With WAPT:
WAPT – website performance tool performs test by emulating activity of many virtual users. Each virtual user can have its own profile settings. You can have thousands of virtual users acting simultaneously on your website performing any activity like reading or writing with your web server. Once you set number of virtual users to act on your website you have option to run your tests for specified time or specified user sessions.
Analyzing the test report:
Test result consists of charts updated in real time which you can monitor when your tests are running. The final comprehensive report is provided at the end of the tests.
Here are the important parameters to be monitored on the test report:
Error Rate: Failure rate against total number of tests run. The error may be due to the high load on server or due to the network problems and timeouts.
Response Time: Obviously a great parameter to check when you run tests for website performance. This response time indicates time required by server to provide correct reply to the request.
Number of pages per second: Number of page requests successfully completed by server per second.
How to conclude performance tests?
These performance criteria change during each test-pass with different load conditions. You need to conclude what is your acceptable load limit and whether your server is able to serve this load.
E.g.: If you expect your server to handle 100 requests successfully per second then anything below this will be failure of your server which needs to be tackled.
How to Record tests:
WAPT works like any other record and playback tool but the real strength is behind it’s parametrization where you can configure any parameter from website url or user session to act as a real user.Testing with WAPT in simple 5 steps:
Record->Configure->Verify->Execute->Analyze
WAPT uses inline Microsoft internet explorer which is used to record your interaction with website. When you record your test all dynamic parameters are recorded as static values which can be configured later while test execution. You then need to configure each user with different settings like unique sessions, number of virtual users, values for dynamic parameters etc. Once you done with recording and configuration just verify your test if it’s ready to run and then execute performance tests if everything looks ok. Finally analyze reports to decide website performance test as accepted or failed against your set of defined standards. That’s it.
WAPT is available in two versions
- Standard version (Latest WAPT 7.5)
- Professional version of this stress and performance testing tool (Latest WAPT Pro 2.5)
What WAPT Pro can do for you?
- Use several computers to generate load on website
- Measure web server performance in terms of CPU, RAM or network usage
- You can include the execution of a JavaScript code into virtual user profiles.
If you don’t want to specify every parameter manually you can use some technology specific modules to significantly improve your test experience.
Following additional modules can be downloaded and installed along with standard or professional version of WAPT:
- Module for ASP.NET testing
- Module for Adobe Flash testing
- Module for JSON format
Finally, any review can’t be complete without the list of Pros and cons.
WAPT Pros:
- Easy to install – Takes only 5 minutes to install- Easy to use with very short learning curve
- You get run-time reports so that you can decide whether to continue the test or not, saving your big time.
- Detailed test report with graphical representation.
- Supports secure HTTPS protocol.
- 30 days free trial available!
WAPT Cons:
- Only windows platform supported to install this tool. (But you can test your website running under any OS and technology)- No scripting ability
- It’s not free
How to try this tool?
You can download 30 day trial version of WAPT from here.
That being said WAPT makes website load, stress and performance testing super easy.
Over to You!
Which performance testing tool do you use?Ask your queries related to WAPT tool or performance testing in comments below.
Wednesday 29 August 2012
DELIVER HIGH VALUE SOFTWARE FEATURES IN A SHORT TIME PERIOD USING AGILE SCRUM PROCESS
What is agile scrum (sprint) process?
Scrum is a software development process. In today’s rapid world stakeholders want immediate return on their investments. They don’t want to wait for longer periods to get full featured product. As a result, nowadays new software development and testing framework is catching momentum i.e. Scrum approach.In scrum, projects are divided in small features to be developed and tested in specific time-frames called as sprint (small cycles). Features should get developed and tested in specified small time-frames. This agile scrum team is handled by scrum master.
Scrum is an iterative, incremental framework for projects and products or application development. Scrum has become more and more popular software development and testing framework among organizations. Many small to large sized IT companies have started to embrace Scrum framework, as this can create excellent quality products in less time than other traditional methodologies. This framework can save companies both time and money.
Source: ScrumAlliance
Soft Skills for a Scrum Team:
What Soft Skills are required to be a Successful Scrum Team?When we start our regular (Agile) sprints (Cycles of work), we usually find some of the challenges with our team members. These challenges are not part of technical difficulties. It usually occurs with team member’s mindset or their soft skills. Many successful Scrum projects taught us that the success of scrum depends on how team members support whole heartedly towards the Sprint.
Let us discuss some of the pre-requisite soft skills for a Scrum Team.
Team Spirit
Cross functional Team work is at the heart of Scrum. There is no “my work”, “I have finished my work” and “your work”. On a Scrum team we find only “Our work”, “we have completed our Sprint”. Individuals will have helping tendency for sharing technical knowledge. Scrum Members are always available to team members rather than locked away behind closed doors. Scrum Master will always motivate the teams and create a Supporting learning environment. Team will always be sprint-oriented and often discuss smooth run of the sprint. A scrum team’s job is to self-organize around the challenges and management’s job is to remove impediments to self-organization.Communication
Good communication must exist among team members of development team, testing team, business analysts and stake holders. There must be highly collaborative interaction between client and the delivery teams. More client involvement implies more suggestions or changes from the client. It implies more bandwidth for communication.Commitment
Agile Teams needs periodic re-energizing to renew their commitments to their purpose and to each other. Scrum Masters can help by ensuring that the team embraces the concept of whole-team responsibility and whole-team commitment to deliver working software at the end of each sprint. With the whole-team commitment, the team member who has completed his tasks will help the one who has not completed so that hopefully each finishes on time.Problem Solving
Scrum does not simply focus on developing just any type of end product. Instead, the Scrum method allows the team to focus on creating a product that fulfils the customer’s highest value priorities which are defined by product owners.Transparency
Transparency among team members and management gives a real momentum to the scrum team. Scrum Master encourages people to ask for help, surface roadblocks, and give public recognition for those brave enough to do so. At the same time, Scrum Master also understands the time wasted and impact on the team when individuals sit on or ignore problems.Scrum Result
If scrum team follows some of above said soft skills, team velocity will increase significantly. In turn, customers will appreciate the results or updates – and also can react quickly to any potential problems. Team can deliver high value software features in a short time period keeps everyone on top of changing business conditions.If you have queries about agile/scrum/sprint software development and testing process then please ask in comments below.
Monday 27 August 2012
What is the Best Way to Make Developer and QA Relationship Healthy?
Testers the troublemakers
It’s funny, how almost everywhere developers consider testers as the troublemakers. Actually it’s not their fault, no one like to hear faults in his/her own baby. And same thing which we (tester) are doing, of course intention behind that is to deliver quality output to client. Constantly there is bitterness at some point in the game between these two roles. Wonder why? It’s the genre and responsibility of these two roles.
When bug count increases or bugs are severe and it’s causing difficulty for developer to solve that bug, developer get frustrated at the count and even at the person as well. The understanding level between these two roles conflict not only in one place but in many areas.
So how to make a good and understanding relationship between testers and developers?
My experience says that teamwork and friendship are the best solutions. If you could be a good friend of developer then you can challenge him to issues, and for sure that person take it positively and work better. It’s the responsibility of both to ensure that the ultimate output is to work at its best. While the developers should ensure that there are no bugs out of what they develop. The testers should ensure that if there are bugs, those should be given, handled at the correct time and scope, where completion comes in.When you are a QA and working with a team for long time, the relationship between you and developers becomes friendlier. As a team you are able to work together finding defects beforehand, which is appreciated always. Not only that, sitting together in a discussion of designs and solutions can make the developers to be aware of the different issues and areas to improve quality, thus taking the quality mind-set a step further.
As a tester, you find the defects but it’s always good to share some tactics with developers on how to test the application. Maybe, this will help the developers to test better before delivering the product. But this can work only if everyone is co-operative enough to look the final target i.e. “to deliver with quality”.
Let’s share your thoughts:
What do you think is the best way to make developer and QA relationship healthy?Few of my thoughts on this are:
1. Share your strategy with developers. Don’t keep it in mind thinking that you will mark it as an issue at later stage.
2. Try to build friendly relations with developers, so that they can feel comfortable to share anything with you.
3. Keep your issue reporting style positive, it should not hurt someone’s feelings.
You might be a developer or QA, let’s add your thoughts in this discussion. So that our ultimate aim of “delivering quality output” will be achieved together.
Saturday 25 August 2012
WRITE A KILLER SOFTWARE TESTING QA RESUME THAT WILL TURN INTO AN INTERVIEW CALL
Can you write a masterpiece of a software testing resume that will turn into an interview call?
If not, read on. I’m sure after reading this article you will be able
to write a killer flawless software testing and quality assurance resume
that will definitely turn into an interview call.
Your resume is the very first step in any job application process. It’s an opportunity to advertise yourself and demonstrate that you are the best person for the available position. Getting an interview call depends on how you present your skills in resume or CV.
You must stand out from the crowd and writing a good resume is the very first opportunity to do so. Recruiters don’t have time to read all the resumes througly. Your resume will be quickly scanned within 20 to 30 seconds. Yes, you get hardly 20 to 30 seconds to persuade your employer to take the decision if to call you for an interview.
Does that make sense? To make a first good impression on prospective employer you must represent yourself effectively on first page of your resume, rather the first half page of your resume is very important to make or break it.
I see so many candidates pay very little or no attention to write a good resume. They just copy and paste others resume without even bothering to change the interests and hobbies. Remember, no matter how talented you are, if you don’t present your skills properly in resume, no one is going to see your talent.
How to Maximize Your Chances of Getting an Interview Call?
Make sure you have a clearly stated job objective mentioned on top of your resume. Keep it short one or two lines and avoid writing irrelevant cliches. Freshers always needs to keep different versions for different jobs. E.g.: If you are applying for software testing position highlight software testing skills at prominent place in your CV.
What if you don’t have software testing experience?
If you are a experienced software tester then you shouldn’t have any problem writing your project details.
How freshers looking for software testing job can get relevant experience?
1) The answer is simple. Get some experience by working on dummy projects available on internet. Search for online dummy projects (e.g. Inventory management software) and download test software and all available documents. Follow complete testing process like:
2) By adding dummy projects learned from software testing courses:
If you have joined any software testing course to learn manual testing and automation tools then you can put this dummy project experience in your resume, which may range from 1 to 6 months. This way you will have at least some experience to put in your resume rather than keeping the experience section entirely blank. This will be an added advantage from other freshers resumes.
E.g:
1) Learn TestLink test management tool online: TestLink online
You can practice everything on above demo TestLink page. Once you get good hands on experience on TestLink tool you can put this skill in your resume.
2) Search for online version of Bugzilla defect management tool or download and install Bugzilla defect management tool on your home PC. Learn how to add and manage defects in Bugzilla. Once you get basic knowledge of this tool you can add this tool under “Defect management tools” skill section.
This way you can learn many automation tools online.
- Career objective – not more than two lines
- Educational qualification – in reverse chronological order (Latest education first)
- Skill upgrade details – like testing certifications, training, computer networking and System administration skills
- Work experience – in detail for each employer and project
- Interests and significant achievements
- Additional personal information like marital status, Passport details etc not more than 3 details.
1) Keep CV brief but comprehensive in expression
2) Keep in mind – Single spelling error is sufficient to reject your resume. Spell check for twice.
3) CV should be easily readable
4) Make a clear job objective
5) Highlight relevant skills
6) Do not put fake experience or skills
7) Focus on what employer’s need and prepare your resume with relevant skills you posses.
8 ) Always think from employer’s perspective. Think what recruiter will expect from the job position.
9) Avoid table structure. Use tables to mention your qualification and skills only.
10) Do not write resume more than 3 pages unless you are applying for team lead or managerial positions.
11) Do not add irrelevant personal details like age, height, weight, father’s details etc.
12) No need to write ‘Curriculum Vitae’ or ‘Resume’ word at the top of your resume.
13) Do not use word “I” while describing project responsibilities. E.g: Instead of “I wrote test cases..” use “Wrote test cases…)
14) Make sure you write your name, email address and phone number on top of the resume.
15) While writing education always start with recent education first.
16) Write qualification details with columns – Education/Qualification, School/College, Year, Percentage/Grade, Class
17) Write relevant skills and on-job-accomplishments on first page of your resume and work experience, educational details on second page.
Most important – Be ready to explain everything you put in your resume. On request you must present necessary examples to interviewer.
Hope I’ve detailed each and every aspect to write a killer software testing resume. Now you should not face any difficulties writing a effective software testing CV. If you need help, please put your queries in comments.
Your resume is the very first step in any job application process. It’s an opportunity to advertise yourself and demonstrate that you are the best person for the available position. Getting an interview call depends on how you present your skills in resume or CV.
How Much Time Do You Get to Impress Employer?
Software testing market is becoming very competitive and getting the job is even more difficult. For a single QA job positions recruiters are getting hundreds of quality assurance tester resumes.
You must stand out from the crowd and writing a good resume is the very first opportunity to do so. Recruiters don’t have time to read all the resumes througly. Your resume will be quickly scanned within 20 to 30 seconds. Yes, you get hardly 20 to 30 seconds to persuade your employer to take the decision if to call you for an interview.
Does that make sense? To make a first good impression on prospective employer you must represent yourself effectively on first page of your resume, rather the first half page of your resume is very important to make or break it.
I see so many candidates pay very little or no attention to write a good resume. They just copy and paste others resume without even bothering to change the interests and hobbies. Remember, no matter how talented you are, if you don’t present your skills properly in resume, no one is going to see your talent.
How to Make a Great First Impression From Your Resume or CV?
Many candidates write whole story about themselves without thinking what employer’s want. First focus on employer’s need. Read the job openings carefully. Note down all the job requirements. Judge yourself based on these requirements. Prepare list of your skills matching with job requirement and highlight these skill on first page of your resume.How to Maximize Your Chances of Getting an Interview Call?
Make sure you have a clearly stated job objective mentioned on top of your resume. Keep it short one or two lines and avoid writing irrelevant cliches. Freshers always needs to keep different versions for different jobs. E.g.: If you are applying for software testing position highlight software testing skills at prominent place in your CV.
Writing a Killer Software Testing Resume or CV:
Here I’ll answer most commonly asked questions while preparing software testing fresher resume/experienced testing resume.What if you don’t have software testing experience?
If you are a experienced software tester then you shouldn’t have any problem writing your project details.
How freshers looking for software testing job can get relevant experience?
1) The answer is simple. Get some experience by working on dummy projects available on internet. Search for online dummy projects (e.g. Inventory management software) and download test software and all available documents. Follow complete testing process like:
- requirement analysis,
- writing test cases,
- executing test cases,
- logging defects and,
- preparing test reports
2) By adding dummy projects learned from software testing courses:
If you have joined any software testing course to learn manual testing and automation tools then you can put this dummy project experience in your resume, which may range from 1 to 6 months. This way you will have at least some experience to put in your resume rather than keeping the experience section entirely blank. This will be an added advantage from other freshers resumes.
How to write project details in tester/QA resume?
In job experience section write details of projects you worked on. Write project details with following headings:- Project name:
- (Optional) Client name:
- Project description: (Brief project overview in 2-3 sentences)
- Environment: (mention software coding language, testing tools etc.)
- Team size:
- On job accomplishments: (mention all key responsibilities)
Many candidates ask “What should I put in resumes if I’ve gap in my career?”
Don’t hesitate to put the valid reason for any gap in your career. Also you shouldn’t have any problem getting job after gap in your career. There could be thousands of reasons for career gap like – enjoying holiday, relocation, handling family business, skill upgrade, maternity etc. Be honest and I’m sure you will easily convince interviewer about your career gap.On-the-job-accomplishments on first page of your resume:
Convince employer that you have problem solving skill by giving some real time examples from your work experience. Clearly state what was the problem and how you solved that problem at workplace. Prepare some solid examples to support your claims. You can put these examples in your resume also. Also be ready to answer all relevant questions asked by interviewer for your accomplishments. E.g: “When I joined so and so project in my company I saw the work was ad-hock and there wasn’t any standard software testing process. I took initiative building a standard software testing process that fits our project needs. By this streamlined process we managed our time effectively and started concentrating more on main software testing tasks”.Mention relevant modules/subjects you studied
This will matter most for freshers. For software testing positions candidates having computer networking and system administration skills are preferred. If you studied any subject or completed any course related to computer networking and system administration then add it in you resume. If you have Linux/Unix operating system knowledge then put it in relevant-skills section of your resume.Software testing certifications and training:
Software testing certification is an added advantage for all testing and QA positions. Rather, testing certifications like ISTQB, CSTE etc. are mandatory criteria for most of the companies. Always keep learning and equip yourself with necessary tools and skills so that you will never face any job problem in future. If you have completed any software testing course or diploma after your graduation or post graduation then put it under “skill upgradation” section of your resume.How to learn software testing skills to put in resume?
IF you don’t have necessary relevant skills to add in your resume then learn those skills online. Like for software testing jobs learn defect tracking and test management tools. You can get all open source software testing tools online. Download widely used open source tools and start practicing at home.E.g:
1) Learn TestLink test management tool online: TestLink online
You can practice everything on above demo TestLink page. Once you get good hands on experience on TestLink tool you can put this skill in your resume.
2) Search for online version of Bugzilla defect management tool or download and install Bugzilla defect management tool on your home PC. Learn how to add and manage defects in Bugzilla. Once you get basic knowledge of this tool you can add this tool under “Defect management tools” skill section.
This way you can learn many automation tools online.
Sample Software Testing Resume Essential Parts:
- Personal details (Name, email and contact) at the top- Career objective – not more than two lines
- Educational qualification – in reverse chronological order (Latest education first)
- Skill upgrade details – like testing certifications, training, computer networking and System administration skills
- Work experience – in detail for each employer and project
- Interests and significant achievements
- Additional personal information like marital status, Passport details etc not more than 3 details.
Tips for Writing Effective Software Testing Resume:
Software testing resume format tips1) Keep CV brief but comprehensive in expression
2) Keep in mind – Single spelling error is sufficient to reject your resume. Spell check for twice.
3) CV should be easily readable
4) Make a clear job objective
5) Highlight relevant skills
6) Do not put fake experience or skills
7) Focus on what employer’s need and prepare your resume with relevant skills you posses.
8 ) Always think from employer’s perspective. Think what recruiter will expect from the job position.
9) Avoid table structure. Use tables to mention your qualification and skills only.
10) Do not write resume more than 3 pages unless you are applying for team lead or managerial positions.
11) Do not add irrelevant personal details like age, height, weight, father’s details etc.
12) No need to write ‘Curriculum Vitae’ or ‘Resume’ word at the top of your resume.
13) Do not use word “I” while describing project responsibilities. E.g: Instead of “I wrote test cases..” use “Wrote test cases…)
14) Make sure you write your name, email address and phone number on top of the resume.
15) While writing education always start with recent education first.
16) Write qualification details with columns – Education/Qualification, School/College, Year, Percentage/Grade, Class
17) Write relevant skills and on-job-accomplishments on first page of your resume and work experience, educational details on second page.
Most important – Be ready to explain everything you put in your resume. On request you must present necessary examples to interviewer.
Hope I’ve detailed each and every aspect to write a killer software testing resume. Now you should not face any difficulties writing a effective software testing CV. If you need help, please put your queries in comments.
GUI TESTING ON SMART DEVICES
As “First impression is the last”, so GUI
(Graphical User Interface) does matter and creates a lot of difference.
Importance of decent and attractive GUI can be felt more significantly
in smart devices environment where screen size is much small.
GUI testing can be toughest part especially while testing on smart device. You should pay full attention to the GUI while testing on smart devices and surely it is an important task that deserves significant time and resource allocation.
Practical Tips for Testing GUI on Smart Devices:
For me, while testing GUI, all the controls are accused. I raise questions why they are there on the screen and I try to answer these questions. I argue in opposition and favor of the controls one by one and I do all this without discussing with someone else. It is the time when I’m wearing multiple hats, Controls are accused and I’m the Prosecutor , I’m the Defense Lawyer and I’m the Judge and during all this process a control must have valid and solid reasons in its favor to be there on screen and consume space. I suggest you to try it and it will help you to decide which controls to display on the screen.
There also come the situations where you are given an already built GUI to test. In such situations also think about the missing controls, the controls that will add value to the screen and compare their importance with the current ones. If you think you need to make a change go ahead.
Once you have decided which controls will be shown on the screen, think thoroughly about size, style and location of the controls on the screen and more important how user will interact with them?
3 important factors to be considered while testing GUI on Smart Devices:
Size:
There are too many variations in screen sizes and available resolutions. In smart devices especially, controls sizes are not static, they have relation to the available screen size.
While testing, make sure that controls size looks esthetically good and control is completely visible on the screen without any scrolling. Test the GUI on different devices with different screen sizes and resolutions.
Emulators are good for this purpose but nothing matches the real device. So make sure that you test on at least two or three real devices. Also don’t forget to test on landscape and portrait orientations if the device supports it.
Style:
Definitely your application has a specific design. And style of the controls should match with that design. You might have seen many applications where some controls e.g. panels have round edges and text boxes in them have sharp edges. Although this type of issues don’t affect the usability or functionality but still a consistent look of the application helps to build a friendly relation between the application and the user.
Relatively more important thing in style is font on the different pages. Most of the times, we focus the text that is visible in normal situations and ignore the text that appears in specific situations. Success and Failure messages are an example of such type of text.
Another factor, important in style is relation between the font color and the situation in which text is displayed. For example Red color is used for Error messages, Green for success, Yellow for warnings and Blue (now a day occasionally) for hyperlinks.
Location:
Location and position are the two words that are used alternatively and it is interesting that they are further used to convey two different concepts that are explained below.
1. Sometimes it is the area on the screen where a control appears. For example Header is located on Top of the page, Labels are Left Aligned, and Text boxes are Right Aligned etc. Here text in bold are relative positions of the controls
2. Sometimes it is the order of a control among the other controls. For example while getting personal info, First Name is followed by the last name or format of controls to ask for a US address should be in order ZIP, City, State.
For both these situations, make sure that everything is logical and shows a good aesthetic sense.
Forgot something even more important. There are situations where one or more controls appear on more than one screen, in this situation make sure that they appear on same location and in the same order on all the pages.
GUI testing can be toughest part especially while testing on smart device. You should pay full attention to the GUI while testing on smart devices and surely it is an important task that deserves significant time and resource allocation.
Practical Tips for Testing GUI on Smart Devices:
For me, while testing GUI, all the controls are accused. I raise questions why they are there on the screen and I try to answer these questions. I argue in opposition and favor of the controls one by one and I do all this without discussing with someone else. It is the time when I’m wearing multiple hats, Controls are accused and I’m the Prosecutor , I’m the Defense Lawyer and I’m the Judge and during all this process a control must have valid and solid reasons in its favor to be there on screen and consume space. I suggest you to try it and it will help you to decide which controls to display on the screen.
There also come the situations where you are given an already built GUI to test. In such situations also think about the missing controls, the controls that will add value to the screen and compare their importance with the current ones. If you think you need to make a change go ahead.
Once you have decided which controls will be shown on the screen, think thoroughly about size, style and location of the controls on the screen and more important how user will interact with them?
3 important factors to be considered while testing GUI on Smart Devices:
Size:
There are too many variations in screen sizes and available resolutions. In smart devices especially, controls sizes are not static, they have relation to the available screen size.
While testing, make sure that controls size looks esthetically good and control is completely visible on the screen without any scrolling. Test the GUI on different devices with different screen sizes and resolutions.
Emulators are good for this purpose but nothing matches the real device. So make sure that you test on at least two or three real devices. Also don’t forget to test on landscape and portrait orientations if the device supports it.
Style:
Definitely your application has a specific design. And style of the controls should match with that design. You might have seen many applications where some controls e.g. panels have round edges and text boxes in them have sharp edges. Although this type of issues don’t affect the usability or functionality but still a consistent look of the application helps to build a friendly relation between the application and the user.
Relatively more important thing in style is font on the different pages. Most of the times, we focus the text that is visible in normal situations and ignore the text that appears in specific situations. Success and Failure messages are an example of such type of text.
Another factor, important in style is relation between the font color and the situation in which text is displayed. For example Red color is used for Error messages, Green for success, Yellow for warnings and Blue (now a day occasionally) for hyperlinks.
Location:
Location and position are the two words that are used alternatively and it is interesting that they are further used to convey two different concepts that are explained below.
1. Sometimes it is the area on the screen where a control appears. For example Header is located on Top of the page, Labels are Left Aligned, and Text boxes are Right Aligned etc. Here text in bold are relative positions of the controls
2. Sometimes it is the order of a control among the other controls. For example while getting personal info, First Name is followed by the last name or format of controls to ask for a US address should be in order ZIP, City, State.
For both these situations, make sure that everything is logical and shows a good aesthetic sense.
Forgot something even more important. There are situations where one or more controls appear on more than one screen, in this situation make sure that they appear on same location and in the same order on all the pages.
Friday 24 August 2012
TEST APPLICATION SECURITY : WEB AND DESKTOP APPLICATION SECURITY TESTING TECHNIQUES
Software industry has achieved a solid recognition in this age. In
the recent decade, however, cyber-world seems to be even more dominating
and driving force which is shaping up the new forms of almost every
business. Web based ERP systems used today are the best evidence that IT
has revolutionized our beloved global village.
These days, websites are not meant only for publicity or marketing but these have been evolved into the stronger tools to cater complete business needs. Web based Payroll systems, Shopping Malls, Banking, Stock Trade application are not only being used by organizations but are also being sold as products today.
This means that online applications have gained the trust of customers and users regarding their vital feature named as SECURITY. No doubt, the security factor is of primary value for desktop applications too. However, when we talk about web, importance of security increases exponentially. If an online system cannot protect the transaction data, no one will ever think of using it. Security is neither a word in search of its definition yet, nor is it a subtle concept. However, I would like to list some complements of security.
2) An ERP system is not secure if DEO (data entry operator) can generate ‘Reports’
3) An online Shopping Mall has no security if customer’s Credit Card Detail is not encrypted
4) A custom software possess inadequate security if an SQL query retrieves actual passwords of its users
Security Testing Definition:
Now, I present you a simplest definition of Security in my own words. “Security means that authorized access is granted to protected data and unauthorized access is restricted”. So, it has two major aspects; first is protection of data and second one is access to that data. Moreover, whether the application is desktop or web based, security revolves around the two aforementioned aspects. Let us have an overview of security aspects for both desktop and web based software applications.
Desktop and Web Security Testing:
A desktop application should be secure not only regarding its access but also with respect to organization and storage of its data. Similarly, a web application demands even more security with respect to its access, along with data protection. Web developer should make the application immune to SQL Injections, Brute Force Attacks and XSS (cross site scripting). Similarly, if the web application facilitates remote access points then these must be secure too. Moreover, keep in mind that Brute Force Attack is not only related to web applications, desktop software is also vulnerable to this.
I hope this foreword is enough and now let me come to the point. Kindly accept my apology if you so far thought that you are reading about the subject of this article. Though I have briefly explained software Security and its major concerns, but my topic is ‘Security Testing’. In order to know further details of security aspects, kindly refer to – Web application security testing article.
I will now explain how the features of security are implemented in software application and how should these be tested. My focus will be on Whats and Hows of security testing, not of security.
How to Test: In order to test this, thorough testing of all roles and rights should be performed. Tester should create several user accounts with different as well multiple roles. Then he should use the application with the help of these accounts and should verify that every role has access to its own modules, screens, forms and menus only. If tester finds any conflict, he should log a security issue with complete confidence.
So, testing of this aspect is already explained above. The second aspect of data protection is related to how that data is stored in the DB. All the sensitive data must be encrypted to make it secure. Encryption should be strong especially for sensitive data like passwords of user accounts, credit card numbers or other business critical information. Third and last aspect is extension of this second aspect. Proper security measures must be adopted when flow of sensitive or business critical data occurs. Whether this data floats between different modules of same application, or is transmitted to different applications it must be encrypted to make it safe.
How to Test Data Protection: The tester should query the database for ‘passwords’ of user account, billing information of clients, other business critical and sensitive data and should verify that all such data is saved in encrypted form in the DB. Similarly (s)he must verify that between different forms or screens, data is transmitted after proper encryption. Moreover, tester should ensure that the encrypted data is properly decrypted at the destination. Special attention should be paid on different ‘submit’ actions. The tester must verify that when the information is being transmitted between client and server, it is not displayed in the address bar of web browser in understandable format. If any of these verifications fail, the application definitely has security flaw.
How to test Brute-Force Attack: The tester must verify that some mechanism of account suspension is available and is working accurately. (S)He must attempt to login with invalid user IDs and Passwords alternatively to make sure that software application blocks the accounts that continuously attempt login with invalid information. If the application is doing so, it is secure against brute-force attack. Otherwise, this security vulnerability must be reported by the tester.
The above three security aspects should be taken into account for both web and desktop applications while, the following points are related with web based applications only.
How to test SQL Injection and XSS: Tester must ensure that maximum lengths of all input fields are defined and implemented. (S)He should also ensure that defined length of input fields does not accommodate any script input as well as tag input. Both these can be easily tested e.g. if 20 is the maximum length specified for ‘Name’ field; and input string “<p>thequickbrownfoxjumpsoverthelazydog” can verify both these constraints. It should also be verified by the tester that application does not support anonymous access methods. In case any of these vulnerabilities exists, the application is in danger.
How to Test Service Access Points: Let me explain it with the example of stock trading web application; an investor (who wants to purchase the shares) should have access to current and historical data of stock prices. User should be given the facility to download this historical data. This demands that application should be open enough. By accommodating and secure, I mean that application should facilitate investors to trade freely (under the legislative regulations). They may purchase or sale 24/7 and the data of transactions must be immune to any hacking attack. Moreover, a large number of users will be interacting with application simultaneously, so the application should provide enough number access point to entertain all the users.
In some cases these access points can be sealed for unwanted applications or people. This depends upon the business domain of application and its users, e.g. a custom web based Office Management System may recognize its users on the basis of IP Addresses and denies to establish a connection with all other systems (applications) that do not lie in the range of valid IPs for that application.
Tester must ensure that all the inter-network and intra-network access to the application is from trusted applications, machines (IPs) and users. In order to verify that an open access point is secure enough, tester must try to access it from different machines having both trusted and untrusted IP addresses. Different sort of real-time transactions should be tried in a bulk to have a good confidence of application’s performance. By doing so, the capacity of access points of the application will also be observed clearly.
Tester must ensure that the application entertains all the communication requests from trusted IPs and applications only while all the other request are rejected. Similarly, if the application has some open access point, then tester should ensure that it allows (if required) uploading of data by users in secure way. By this secure way I mean, the file size limit, file type restriction and scanning of uploaded file for viruses or other security threats. This is all how a tester can verify the security of an application with respect to its access points.
These days, websites are not meant only for publicity or marketing but these have been evolved into the stronger tools to cater complete business needs. Web based Payroll systems, Shopping Malls, Banking, Stock Trade application are not only being used by organizations but are also being sold as products today.
This means that online applications have gained the trust of customers and users regarding their vital feature named as SECURITY. No doubt, the security factor is of primary value for desktop applications too. However, when we talk about web, importance of security increases exponentially. If an online system cannot protect the transaction data, no one will ever think of using it. Security is neither a word in search of its definition yet, nor is it a subtle concept. However, I would like to list some complements of security.
Examples of security flaws in an application:
1) A Student Management System is insecure if ‘Admission’ branch can edit the data of ‘Exam’ branch2) An ERP system is not secure if DEO (data entry operator) can generate ‘Reports’
3) An online Shopping Mall has no security if customer’s Credit Card Detail is not encrypted
4) A custom software possess inadequate security if an SQL query retrieves actual passwords of its users
Security Testing Definition:
Now, I present you a simplest definition of Security in my own words. “Security means that authorized access is granted to protected data and unauthorized access is restricted”. So, it has two major aspects; first is protection of data and second one is access to that data. Moreover, whether the application is desktop or web based, security revolves around the two aforementioned aspects. Let us have an overview of security aspects for both desktop and web based software applications.
Desktop and Web Security Testing:
A desktop application should be secure not only regarding its access but also with respect to organization and storage of its data. Similarly, a web application demands even more security with respect to its access, along with data protection. Web developer should make the application immune to SQL Injections, Brute Force Attacks and XSS (cross site scripting). Similarly, if the web application facilitates remote access points then these must be secure too. Moreover, keep in mind that Brute Force Attack is not only related to web applications, desktop software is also vulnerable to this.
I hope this foreword is enough and now let me come to the point. Kindly accept my apology if you so far thought that you are reading about the subject of this article. Though I have briefly explained software Security and its major concerns, but my topic is ‘Security Testing’. In order to know further details of security aspects, kindly refer to – Web application security testing article.
I will now explain how the features of security are implemented in software application and how should these be tested. My focus will be on Whats and Hows of security testing, not of security.
Security Testing Techniques:
1) Access to Application:
Whether it is a desktop application of website, access security is implemented by ‘Roles and Rights Management’. It is often done implicitly while covering functionality, e.g.in a Hospital Management System a receptionist is least concerned about the laboratory tests as his job is to just register the patients and schedule their appointments with doctors. So, all the menus, forms and screen related to lab tests will not be available to the Role of ‘Receptionist’. Hence, the proper implementation of roles and rights will guarantee the security of access.How to Test: In order to test this, thorough testing of all roles and rights should be performed. Tester should create several user accounts with different as well multiple roles. Then he should use the application with the help of these accounts and should verify that every role has access to its own modules, screens, forms and menus only. If tester finds any conflict, he should log a security issue with complete confidence.
2. Data Protection:
There are further three aspects of data security. First one is that a user can view or utilize only the data which he is supposed to use. This is also ensured by roles and rights e.g. a TSR (telesales representative) of a company can view the data of available stock, but cannot see how much raw material was purchased for production.So, testing of this aspect is already explained above. The second aspect of data protection is related to how that data is stored in the DB. All the sensitive data must be encrypted to make it secure. Encryption should be strong especially for sensitive data like passwords of user accounts, credit card numbers or other business critical information. Third and last aspect is extension of this second aspect. Proper security measures must be adopted when flow of sensitive or business critical data occurs. Whether this data floats between different modules of same application, or is transmitted to different applications it must be encrypted to make it safe.
How to Test Data Protection: The tester should query the database for ‘passwords’ of user account, billing information of clients, other business critical and sensitive data and should verify that all such data is saved in encrypted form in the DB. Similarly (s)he must verify that between different forms or screens, data is transmitted after proper encryption. Moreover, tester should ensure that the encrypted data is properly decrypted at the destination. Special attention should be paid on different ‘submit’ actions. The tester must verify that when the information is being transmitted between client and server, it is not displayed in the address bar of web browser in understandable format. If any of these verifications fail, the application definitely has security flaw.
3. Brute-Force Attack:
Brute Force Attack is mostly done by some software tools. The concept is that using a valid user ID, software attempts to guess the associated password by trying to login again and again. A simple example of security against such attack is account suspension for a short period of time as all the mailing applications like ‘Yahoo’ and ‘Hotmail’ do. If, a specific number of consecutive attempts (mostly 3) fail to login successfully, then that account is blocked for some time (30 minutes to 24 hrs).How to test Brute-Force Attack: The tester must verify that some mechanism of account suspension is available and is working accurately. (S)He must attempt to login with invalid user IDs and Passwords alternatively to make sure that software application blocks the accounts that continuously attempt login with invalid information. If the application is doing so, it is secure against brute-force attack. Otherwise, this security vulnerability must be reported by the tester.
The above three security aspects should be taken into account for both web and desktop applications while, the following points are related with web based applications only.
4. SQL Injection and XSS (cross site scripting):
Conceptually speaking, the theme of both these hacking attempts is similar, so these are discussed together. In this approach, malicious script is used by the hackers in order to manipulate a website. There are several ways to immune against such attempts. For all input fields of the website, field lengths should be defined small enough to restrict input of any script e.g. Last Name should have field length 30 instead of 255. There may be some input fields where large data input is necessary, for such fields proper validation of input should be performed prior to saving that data in the application. Moreover, in such fields any html tags or script tag input must be prohibited. In order to provoke XSS attacks, the application should discard script redirects from unknown or untrusted applications.How to test SQL Injection and XSS: Tester must ensure that maximum lengths of all input fields are defined and implemented. (S)He should also ensure that defined length of input fields does not accommodate any script input as well as tag input. Both these can be easily tested e.g. if 20 is the maximum length specified for ‘Name’ field; and input string “<p>thequickbrownfoxjumpsoverthelazydog” can verify both these constraints. It should also be verified by the tester that application does not support anonymous access methods. In case any of these vulnerabilities exists, the application is in danger.
5. Service Access Points (Sealed and Secure Open)
Today, businesses depend and collaborate with each other, same holds good for applications especially websites. In such case, both the collaborators should define and publish some access points for each other. So far the scenario seems quite simple and straightforward but, for some web based product like stock trading, things are not so simple and easy. When there is large number of target audience, the access points should be open enough to facilitate all users, accommodating enough to fulfill all users’ requests and secure enough to cope with any security-trial.How to Test Service Access Points: Let me explain it with the example of stock trading web application; an investor (who wants to purchase the shares) should have access to current and historical data of stock prices. User should be given the facility to download this historical data. This demands that application should be open enough. By accommodating and secure, I mean that application should facilitate investors to trade freely (under the legislative regulations). They may purchase or sale 24/7 and the data of transactions must be immune to any hacking attack. Moreover, a large number of users will be interacting with application simultaneously, so the application should provide enough number access point to entertain all the users.
In some cases these access points can be sealed for unwanted applications or people. This depends upon the business domain of application and its users, e.g. a custom web based Office Management System may recognize its users on the basis of IP Addresses and denies to establish a connection with all other systems (applications) that do not lie in the range of valid IPs for that application.
Tester must ensure that all the inter-network and intra-network access to the application is from trusted applications, machines (IPs) and users. In order to verify that an open access point is secure enough, tester must try to access it from different machines having both trusted and untrusted IP addresses. Different sort of real-time transactions should be tried in a bulk to have a good confidence of application’s performance. By doing so, the capacity of access points of the application will also be observed clearly.
Tester must ensure that the application entertains all the communication requests from trusted IPs and applications only while all the other request are rejected. Similarly, if the application has some open access point, then tester should ensure that it allows (if required) uploading of data by users in secure way. By this secure way I mean, the file size limit, file type restriction and scanning of uploaded file for viruses or other security threats. This is all how a tester can verify the security of an application with respect to its access points.
DATABASE TESTING
As a tester, you have to test the ‘Examination Results’ module of the
website of a university. Consider the whole application has been
integrated and it is in ‘Ready for Testing’ state. ‘Examination Module’
is linked with ‘Registration’, ‘Courses’ and ‘Finance’ modules. Assume
that you have adequate information of the
application and you created a comprehensive list of test scenarios. Now
you have to design, document and execute these test cases. In
‘Actions/Steps’ section of the test cases, you must mention the
acceptable data as input for the test. The data mentioned in test cases
must be selected properly. The accuracy of ‘Actual Results’ column of TC
Document is primarily dependent upon the test data. So, step to prepare
the input test data is significantly important. Thus, here is my
rundown on ”DB Testing – Test Data Preparation Strategies”.
The test data should be selected precisely and it must possess the following four qualities:
1. Realistic: By realistic, it means the data should be accurate in the context of real life e.g. in order to test ‘Age’ field, all the values should be positive and 18 or above. It is quite obvious that the candidates for an admission in the university are usually 18 years old (this might be defined in requirements).
2. Practically valid: This is similar to realistic but not the same. This property is more related to the business logic of AUT e.g. value 60 is realistic in age field but practically invalid for a candidate of Graduation or even Masters Programs. In this case, a valid range would be 18-25 years (this might be defined in requirements).
3. Versatile to cover scenarios: There may be several subsequent conditions in a single scenario, so choose the data shrewdly to cover maximum aspects of a single scenario with minimum set of data, e.g. while creating test data for result module, do not only consider the case of regular students who are smoothly completing their program. Give attention to the students who are repeating the same course and belong to different semesters or even different programs. The data set may look like this:
There might be several other interesting and tricky sub-conditions.
E.g. the limitation of years to complete a degree program, passing a
prerequisite course for registering a course, maximum no. of courses a
student may enroll in a single semester etc. etc. Make sure to cover all
these scenarios wisely with finite set of data.
4. Exceptional data (if applicable/required): There may be certain exceptional scenarios that are less frequent but demand high importance when occur, e.g. disabled students related issues.
There are only two ways to prepare test data:
Method 1. Insert New Data:
Get a clean DB and insert all the data as specified in your test cases. Once, all your required and desired data has been entered, start executing your test cases and fill ‘Pass/Fail’ columns by comparing the ‘Actual Output’ with ‘Expected Output’. Sounds simple, right? But wait, it’s not that simple.
Few essential and critical concerns are as follows:
This is the feasible and more practical technique for test data preparation. However it requires sound technical skills and demands detailed knowledge of DB Schema and SQL. In this method you need to copy and use production data by replacing some field values by dummy values. This is the best data subset for your testing as it represents the production data. But this may not be feasible all the time due to data security and privacy issues.
Properties of Test Data:
1. Realistic: By realistic, it means the data should be accurate in the context of real life e.g. in order to test ‘Age’ field, all the values should be positive and 18 or above. It is quite obvious that the candidates for an admission in the university are usually 18 years old (this might be defined in requirements).
2. Practically valid: This is similar to realistic but not the same. This property is more related to the business logic of AUT e.g. value 60 is realistic in age field but practically invalid for a candidate of Graduation or even Masters Programs. In this case, a valid range would be 18-25 years (this might be defined in requirements).
3. Versatile to cover scenarios: There may be several subsequent conditions in a single scenario, so choose the data shrewdly to cover maximum aspects of a single scenario with minimum set of data, e.g. while creating test data for result module, do not only consider the case of regular students who are smoothly completing their program. Give attention to the students who are repeating the same course and belong to different semesters or even different programs. The data set may look like this:
| Sr# | Student_ID | Program_ID | Course_ID | Grade |
| 1 | BCS-Fall2011-Morning-01 | BCS-F11 | CS-401 | A |
| 2 | BCS-Spring2011-Evening-14 | BCS-S11 | CS-401 | B+ |
| 3 | MIT-Fall2010-Afternoon-09 | MIT-F10 | CS-401 | A- |
| … | … | … | … | … |
4. Exceptional data (if applicable/required): There may be certain exceptional scenarios that are less frequent but demand high importance when occur, e.g. disabled students related issues.
Test data preparation techniques:
We have briefly discussed the important properties of test data and it also elaborates how test data selection is important while database testing. Now let’s discuss the ‘techniques to prepare test data’.There are only two ways to prepare test data:
Method 1. Insert New Data:
Get a clean DB and insert all the data as specified in your test cases. Once, all your required and desired data has been entered, start executing your test cases and fill ‘Pass/Fail’ columns by comparing the ‘Actual Output’ with ‘Expected Output’. Sounds simple, right? But wait, it’s not that simple.
Few essential and critical concerns are as follows:
- Empty instance of database may not be available
- Inserted test data may be insufficient for testing some cases like performance and load testing.
- Inserting the required test data into blank DB is not an easy job due to the database table dependencies. Because of this inevitable restriction, data insertion can become difficult task for tester.
- Insertion of limited test data (just according to the test cases needs) may hide some issues that could be found only with the large data set.
- For data insertion, complex queries and/or procedures may be required, and for this sufficient assistance or help from the DB developer(s) would be necessary.
- Execution of TCs becomes more efficient as the DB has the required data only.
- Bugs isolation requires no time as only the data specified in test cases present in the DB.
- Less time required for testing and results comparison.
- Clutter-free test process
This is the feasible and more practical technique for test data preparation. However it requires sound technical skills and demands detailed knowledge of DB Schema and SQL. In this method you need to copy and use production data by replacing some field values by dummy values. This is the best data subset for your testing as it represents the production data. But this may not be feasible all the time due to data security and privacy issues.
ACHIEVE LEVEL 5 MATURITY FOR QA AND TESTING PROCESS
For any process whether it is a QA process, development process or
any non-technical process, there are levels of its maturity. By levels
of maturity we mean that the level of formality and processes
improvement, like ad-hoc processes – to formally defined steps – to
managed result metrics – to optimization of the processes.
CMM (Capability Maturity Model) is process based model which is used to assess the maturity of an organization for different domains. Although this model is normally termed as the software development model but eventually it was used for other processes as well like QA and testing.
It has 5 different levels of maturity from 1 to 5. As we go towards level 5 from 1, variability and inconsistency reduces. Below are the details of 5 levels. Here we will go through the 5 CMM levels with respect to QA process and what all output/result is expected for each level to mature a QA/testing process and reach up to level 5.
As there are no pre-defined standards and processes, same task is done in different ways by different people.
And this becomes even more unsystematic and inconsistent if same task is done differently next time.
Example -
QA – The example would be that in an organization although QA is 1 of the phases in a product life cycle but there are not any standard & no process defined, no templates for QA deliverables like plan, strategy, scenarios, and cases are standardized. Even if these are documented then all team members have their own way of doing it and not consistent at all.
Example -
QA – Define overall QA process and methodology for different types of testing like functional, data, performance etc. Define the role of a QA engineer in project’s life cycle and prepare templates for deliverables in each phase. Not only define and prepare rather share within team
Example -
QA – Conduct webinars and training sessions to let people get acquainted about the newly defined QA process and standards and motivate them to make use of those during their day to day project’s life
Example -
QA – Performing regular audits would be a good idea here. This can include to check if teams are actually following the processes defined, using the standard templates, adhere to methodology or not.
Example -
QA – Keep on improving the methodology, processes defined based on prior audit results.
Based on some studies it has been concluded that the organizations at level 1 may spend $1000 for any particular task then for the same task organization at level 5 needs to spend $10.
After going though all 5 levels mentioned above, looks like reaching up to level 3 is difficult. Once it achieved then next levels are not too far and difficult to achieve
CMM (Capability Maturity Model) is process based model which is used to assess the maturity of an organization for different domains. Although this model is normally termed as the software development model but eventually it was used for other processes as well like QA and testing.
It has 5 different levels of maturity from 1 to 5. As we go towards level 5 from 1, variability and inconsistency reduces. Below are the details of 5 levels. Here we will go through the 5 CMM levels with respect to QA process and what all output/result is expected for each level to mature a QA/testing process and reach up to level 5.
Level 1 – Ad-Hoc: Unplanned, unsystematic, and inconsistent
As the word ‘Ad-Hoc’ states: unplanned, unprepared, at this level significance is not given to planning, following processes, guidelines and standards. There is no standardized & consistent way of doing any task. The only thing which is important at this level is meeting the timelines, irrespective of the quality of the end product and deliverables.As there are no pre-defined standards and processes, same task is done in different ways by different people.
And this becomes even more unsystematic and inconsistent if same task is done differently next time.
Example -
QA – The example would be that in an organization although QA is 1 of the phases in a product life cycle but there are not any standard & no process defined, no templates for QA deliverables like plan, strategy, scenarios, and cases are standardized. Even if these are documented then all team members have their own way of doing it and not consistent at all.
Level 2 – Control: initiate defining processes at high level:
Solution to the problem which we saw at Level 1 of unavailability of QA processes, methodology & standards would be to have all these in place. The standards and processes are not only finalized but also are well documented, so that those can be re-used by any one for similar task.Example -
QA – Define overall QA process and methodology for different types of testing like functional, data, performance etc. Define the role of a QA engineer in project’s life cycle and prepare templates for deliverables in each phase. Not only define and prepare rather share within team
Level 3 – Core Competency: Come up with a generalized process for wider audience and domains:
At this level 3, people are motivated to follow the standards and processes defined at level 2. For this first of all the processes need to be conveyed to all people and need to identify what all skills are needed to use those effectively and efficiently and also if any training is required for that and then motivated and supported to follow those standards and processes. Here people having more experience share their knowledge with others.Example -
QA – Conduct webinars and training sessions to let people get acquainted about the newly defined QA process and standards and motivate them to make use of those during their day to day project’s life
Level 4 – Predictable: Measure the processes
At this level processes defined at level 3 are measured quantitatively. This is done to control the effort required on any task. Based on this quantitative analysis, processes can be adjusted if needed, and that to without degrading the quality of the end product. Analysis is done by dividing complete process into smaller sub-processes and then quantitative techniques are applied on these sub-processes and as per the result, sub-processes are adjusted if needed. This level is called predictable as based on prior experience; we can predict the process quantitatively and make use of that for the upcoming processes.Example -
QA – Performing regular audits would be a good idea here. This can include to check if teams are actually following the processes defined, using the standard templates, adhere to methodology or not.
Level 5 – Innovative: Continuous Improvement
At this level, innovative ways are identified to further improve the pre-defined processes and standards. This is a continuous process. For this our own processes are watched and re-engineered continuously by adding new tools technologies, by continuous studies and by keeping ourselves updated with new information in the market. This can also be achieved by benchmarking other organizations and learn from them and try to improve our process by adding new innovations to it.Example -
QA – Keep on improving the methodology, processes defined based on prior audit results.
Based on some studies it has been concluded that the organizations at level 1 may spend $1000 for any particular task then for the same task organization at level 5 needs to spend $10.
After going though all 5 levels mentioned above, looks like reaching up to level 3 is difficult. Once it achieved then next levels are not too far and difficult to achieve
HOW TO BUILD AND GROW YOUR QA TEAM
Like in any other software development life cycle, Testing too requires some important factors to develop and maintain for continuous process improvement. One such factor is Team Building. While building a right team, focus should be on the following key elements:
Roles and Responsibilities
It is very important for the team members to understand what they are supposed to do. This was quite often not communicated or discussed with the team. Before start of a project, the team members must be explained on the typical tasks which they will be performing on a daily basis for their respective roles. Be it a tester or a test lead, setting the expectations and explaining what is expected out of them will give correct results without unnecessary delays or errors.
Following points need to be clarified to the team:
- Scope of the Project
- Roles and Responsibilities expected from everyone
- Key points to focus like Deliverables, Timelines etc.
- Explain about the Strategy and Plan
Knowledge Transfer
It is very vital for the Testers to understand the Domain as well as the functions of the application to be able to thoroughly test the application under test. KT sessions are very essential to make them understand the core functions and logics, which will be used to apply during testing. Brainstorming sessions are vital to share common understanding of application and domain.Discussion should involve testers right from the project initial discussions which essentially consists of Business people, Architects, Developers, Database experts etc.. Involving testers during these early stages of software development will provide good knowledge and understanding about the application that is going to be developed and tested.
Domain Knowledge
Understanding the application’s Domain (e.g. Healthcare, Insurance etc) is very important and will be helpful for Testers to verify the functionality with a different perspective, wearing the hat of the end customer as well as a SME. It takes time and only over the period of working in a particular domain, the resource will be able to familiarize on the domain he is working. Sometimes, a tester will get a chance to test different applications belonging to the same domain, so testing becomes easier and meaningful if he has knowledge on the overall domain.Technical and Domain Certifications
Having a talented pool of testers is definitely a big asset for the project. The focus should be to train the team and get them certified in the respective areas they work by nominating for internal certifications. There are also a host of external certifications which can also be selected and get the team trained.Certifications will definitely give the team a moral support and maturity to perform testing with confidence. Domain certified resources will also leverage the intellectual knowledge gain which can be showcased to prospective clients for new business opportunities.
Career Ladder
It’s not enough to create just a team of testers with all skill set, but to provide opportunities for them to come up their career ladder is also significantly important. Create or nominate to programs to shape them eligible for their next level of role will obviously fulfil the needs of identifying resources when required. Team meetings can be effectively utilized to emphasize their roles and responsibilities in their next level. Educating them the various skills required to perform in their next roles is a very good advantage and also a continuous process improvement. Every Manager has the responsibility to explain about the duties that are expected to be performed when the resources are getting promoted. This will make sure that not just a set of resources are promoted, but a ready-to-work responsible and skilled individuals are.Team Dynamics and Group Outing
It’s quite obvious to ensure there is a level of team dynamics established and followed by the team for effective group work, meeting common goals, finishing panned targets and achieving on time. Making them understand that “Project” is the common objective for all in the project and completing what the customer wants is “Priority”. To accomplish this, everyone should work together as a “Team” leaving all differences behind and completing the planned tasks is the only “Target”. During weekly team meetings, the team members should receive the information on Tasks, Priorities for the next period and have common understanding on the work to be performed, clear and loud.Team building exercises and outings are really necessary to burn out the stress and for a good recharge. This will also help for a better understanding outside the project works and in a different environment altogether. Small token of appreciation can be announced during team meetings to identify talents and to encourage and motivate others to perform.
PENETRATION TESTING
What is Penetration Testing?
It’s the process to identify security vulnerabilities in an application by evaluating the system or network with various malicious techniques. Purpose of this test is to secure important data from outsiders like hackers who can have unauthorized access to system. Once vulnerability is identified it is used to exploit system in order to gain access to sensitive information.
Causes of vulnerabilities:
- Design and development errors
- Poor system configuration
- Human errors
- Many clients are asking for pen testing as part of the software release cycle
- To secure user data
- To find security vulnerabilities in an application
It’s very important for any organization to identify security issues present in internal network and computers. Using this information organization can plan defense against any hacking attempt. User privacy and data security are the biggest concerns nowadays. Imagine if any hacker manage to get user details of social networking site like Facebook. Organization can face legal issues due to a small loophole left in a software system. Hence big organizations are looking for PCI compliance certifications before doing any business with third party clients.
What should be tested?
- Software
- Hardware
- Network
- Process
2) Application Security Testing: Using software methods one can verify if the system is exposed to security vulnerabilities.
3) Physical Penetration Test: Strong physical security methods are applied to protect sensitive data. This is generally useful in military and government facilities. All physical network devices and access points are tested for possibilities of any security breach.
Pen Testing Techniques:
1) Manual penetration test
2) Using automated penetration test tools
3) Combination of both manual and automated process
The third process is more common to identify all kinds of vulnerabilities.
Criteria to select the best penetration tool:
- It should be easy to deploy, configure and use.
- It should scan your system easily.
- It should categorize vulnerabilities based on severity that needs immediate fix.
- It should be able to automate verification of vulnerabilities.
- It should re-verify exploits found previously.
- It should generate detailed vulnerability reports and logs.
Once you know what tests you need to perform you can either train your internal test resources or hire expert consultants to do the penetration task for you.
Examples of Free and Commercial Tools -
Nmap, Nessus, Metasploit, Wireshark, OpenSSL, Cain & Abel, THC Hydra, w3af
Commercial services: Pure Hacking, Torrid Networks, SecPoint, Veracode.
Limitations of Pentest tools: Sometimes these tools can flag false positive output which results in spending more developer time on analyzing such vulnerabilities which are not present.
Penetration Test Process:
Let’s discuss the actual process followed by test agencies or penetration testers. Identifying vulnerabilities present in system is the first important step in this process. Corrective action is taken on these vulnerability and same penetration tests are repeated until system is negative to all those tests.
We can categorize this process in following methods:
1) Data collection: Various methods including Google search are used to get target system data. One can also use web page source code analysis technique to get more info about the system, software and plugin versions. There are many free tools and services available in the market which can give you information like database or table names, DB versions, software versions, hardware used and various third party plugins used in the target system.
2) Vulnerability Assessment: Based on the data collected in first step one can find the security weakness in the target system. This helps penetration testers to launch attacks using identified entry points in the system.
3) Actual Exploit: This is crucial step. It requires special skills and techniques to launch attack on target system. Experienced penetration testers can use their skills to launch attack on the system.
4) Result analysis and report preparation: After completion of penetration tests detailed reports are prepared for taking corrective actions. All identified vulnerabilities and recommended corrective methods are listed in these reports. You can customize vulnerability report format (HTML, XML, MS Word or PDF) as per your organization needs.
1) Check if web application is able to identify spam attacks on contact forms used in the website.
2) Proxy server – Check if network traffic is monitored by proxy appliances. Proxy server make it difficult for hackers to get internal details of the network thus protecting the system from external attacks.
3) Spam email filters – Verify if incoming and outgoing email traffic is filtered and unsolicited emails are blocked. Many email clients come with in-build spam filters which needs to be configured as per your needs. These configuration rules can be applied on email headers, subject or body.
4) Firewall – Make sure entire network or computers are protected with Firewall. Firewall can be a software or hardware to block unauthorized access to system. Firewall can prevent sending data outside the network without your permission.
5) Try to exploit all servers, desktop systems, printers and network devices.
6) Verify that all usernames and passwords are encrypted and transferred over secured connection like https.
7) Verify information stored in website cookies. It should not be in readable format.
8 ) Verify previously found vulnerabilities to check if the fix is working.
9) Verify if there is no open port in network.
11) Verify all telephone devices.
12) Verify WIFI network security.
13) Verify all HTTP methods. PUT and Delete methods should not be enabled on web server .
14) Password should be at least 8 character long containing at least one number and one special character.
15) Username should not be like “admin” or “administrator”.
16) Application login page should be locked upon few unsuccessful login attempts.
17) Error messages should be generic and should not mention specific error details like “Invalid username” or “Invalid password”.
19) Verify if special characters, html tags and scripts are handled properly as an input value.
20) Internal system details should not be revealed in any of the error or alert messages.
21) Custom error messages should be displayed to end user in case of web page crash.
22) Verify use of registry entries. Sensitive information should not be kept in registry.
23) All files must be scanned before uploading to server.
24) Sensitive data should not be passed in urls while communicating with different internal modules of the web application.
25) There should not be any hard coded username or password in the system.
26) Verify all input fields with long input string with and without spaces.
27) Verify if reset password functionality is secure.
28) Verify application for SQL Injection.
29) Verify application for Cross Site Scripting.
31) Important input validations should be done at server side instead of JavaScript checks at client side.
32) Critical resources in the system should be available to authorized persons and services only.
33) All access logs should be maintained with proper access permissions.
34) Verify user session ends upon log off.
35) Verify that directory browsing is disabled on server.
36) Verify that all applications and database versions are up to date.
37) Verify url manipulation to check if web application is not showing any unwanted information.
38) Verify memory leak and buffer overflow.
39) Verify if incoming network traffic is scanned to find Trojan attacks.
40) Verify if system is safe from Brute Force Attacks – a trial and error method to find sensitive information like passwords.
41) Verify if system or network is secured from DoS (denial-of-service) attacks. Hacker can target network or single computer with continuous requests due to which resources on target system gets overloaded resulting in denial of service for legit requests.
These are just the basic test scenarios to get started with Pentest. There are hundreds of advanced penetration methods which can be done either manually or with the help of automation tools.
It’s the process to identify security vulnerabilities in an application by evaluating the system or network with various malicious techniques. Purpose of this test is to secure important data from outsiders like hackers who can have unauthorized access to system. Once vulnerability is identified it is used to exploit system in order to gain access to sensitive information.
Causes of vulnerabilities:
- Design and development errors
- Poor system configuration
- Human errors
Why Penetration testing?
- Financial data must be secured while transferring between different systems- Many clients are asking for pen testing as part of the software release cycle
- To secure user data
- To find security vulnerabilities in an application
It’s very important for any organization to identify security issues present in internal network and computers. Using this information organization can plan defense against any hacking attempt. User privacy and data security are the biggest concerns nowadays. Imagine if any hacker manage to get user details of social networking site like Facebook. Organization can face legal issues due to a small loophole left in a software system. Hence big organizations are looking for PCI compliance certifications before doing any business with third party clients.
What should be tested?
- Software
- Hardware
- Network
- Process
Penetration Testing Types:
1) Social Engineering: Human errors are the main causes of security vulnerability. Security standards and policies should be followed by all staff members to avoid social engineering penetration attempt. Example of these standards include not to mention any sensitive information in email or phone communication. Security audits can be conducted to identify and correct process flaws.2) Application Security Testing: Using software methods one can verify if the system is exposed to security vulnerabilities.
3) Physical Penetration Test: Strong physical security methods are applied to protect sensitive data. This is generally useful in military and government facilities. All physical network devices and access points are tested for possibilities of any security breach.
Pen Testing Techniques:
1) Manual penetration test
2) Using automated penetration test tools
3) Combination of both manual and automated process
The third process is more common to identify all kinds of vulnerabilities.
Penetration Testing Tools:
Automated tools can be used to identify some standard vulnerability present in an application. Pentest tools scan code to check if there is malicious code present which can lead to potential security breach. Pentest tools can verify security loopholes present in the system like data encryption techniques and hard coded values like username and password.Criteria to select the best penetration tool:
- It should be easy to deploy, configure and use.
- It should scan your system easily.
- It should categorize vulnerabilities based on severity that needs immediate fix.
- It should be able to automate verification of vulnerabilities.
- It should re-verify exploits found previously.
- It should generate detailed vulnerability reports and logs.
Once you know what tests you need to perform you can either train your internal test resources or hire expert consultants to do the penetration task for you.
Examples of Free and Commercial Tools -
Nmap, Nessus, Metasploit, Wireshark, OpenSSL, Cain & Abel, THC Hydra, w3af
Commercial services: Pure Hacking, Torrid Networks, SecPoint, Veracode.
Limitations of Pentest tools: Sometimes these tools can flag false positive output which results in spending more developer time on analyzing such vulnerabilities which are not present.
Manual Penetration Test:
It’s difficult to find all vulnerabilities using automated tools. There are some vulnerabilities which can be identified by manual scan only. Penetration testers can perform better attacks on application based on their skills and knowledge of system being penetrated. The methods like social engineering can be done by humans only. Manual checking includes design, business logic as well as code verification.Penetration Test Process:
Let’s discuss the actual process followed by test agencies or penetration testers. Identifying vulnerabilities present in system is the first important step in this process. Corrective action is taken on these vulnerability and same penetration tests are repeated until system is negative to all those tests.
We can categorize this process in following methods:
1) Data collection: Various methods including Google search are used to get target system data. One can also use web page source code analysis technique to get more info about the system, software and plugin versions. There are many free tools and services available in the market which can give you information like database or table names, DB versions, software versions, hardware used and various third party plugins used in the target system.
2) Vulnerability Assessment: Based on the data collected in first step one can find the security weakness in the target system. This helps penetration testers to launch attacks using identified entry points in the system.
3) Actual Exploit: This is crucial step. It requires special skills and techniques to launch attack on target system. Experienced penetration testers can use their skills to launch attack on the system.
4) Result analysis and report preparation: After completion of penetration tests detailed reports are prepared for taking corrective actions. All identified vulnerabilities and recommended corrective methods are listed in these reports. You can customize vulnerability report format (HTML, XML, MS Word or PDF) as per your organization needs.
Penetration testing sample test cases (test scenarios):
Remember this is not functional testing. In Pentest your goal is to find security holes in the system. Below are some generic test cases and not necessarily applicable for all applications.1) Check if web application is able to identify spam attacks on contact forms used in the website.
2) Proxy server – Check if network traffic is monitored by proxy appliances. Proxy server make it difficult for hackers to get internal details of the network thus protecting the system from external attacks.
3) Spam email filters – Verify if incoming and outgoing email traffic is filtered and unsolicited emails are blocked. Many email clients come with in-build spam filters which needs to be configured as per your needs. These configuration rules can be applied on email headers, subject or body.
4) Firewall – Make sure entire network or computers are protected with Firewall. Firewall can be a software or hardware to block unauthorized access to system. Firewall can prevent sending data outside the network without your permission.
5) Try to exploit all servers, desktop systems, printers and network devices.
6) Verify that all usernames and passwords are encrypted and transferred over secured connection like https.
7) Verify information stored in website cookies. It should not be in readable format.
8 ) Verify previously found vulnerabilities to check if the fix is working.
9) Verify if there is no open port in network.
11) Verify all telephone devices.
12) Verify WIFI network security.
13) Verify all HTTP methods. PUT and Delete methods should not be enabled on web server .
14) Password should be at least 8 character long containing at least one number and one special character.
15) Username should not be like “admin” or “administrator”.
16) Application login page should be locked upon few unsuccessful login attempts.
17) Error messages should be generic and should not mention specific error details like “Invalid username” or “Invalid password”.
19) Verify if special characters, html tags and scripts are handled properly as an input value.
20) Internal system details should not be revealed in any of the error or alert messages.
21) Custom error messages should be displayed to end user in case of web page crash.
22) Verify use of registry entries. Sensitive information should not be kept in registry.
23) All files must be scanned before uploading to server.
24) Sensitive data should not be passed in urls while communicating with different internal modules of the web application.
25) There should not be any hard coded username or password in the system.
26) Verify all input fields with long input string with and without spaces.
27) Verify if reset password functionality is secure.
28) Verify application for SQL Injection.
29) Verify application for Cross Site Scripting.
31) Important input validations should be done at server side instead of JavaScript checks at client side.
32) Critical resources in the system should be available to authorized persons and services only.
33) All access logs should be maintained with proper access permissions.
34) Verify user session ends upon log off.
35) Verify that directory browsing is disabled on server.
36) Verify that all applications and database versions are up to date.
37) Verify url manipulation to check if web application is not showing any unwanted information.
38) Verify memory leak and buffer overflow.
39) Verify if incoming network traffic is scanned to find Trojan attacks.
40) Verify if system is safe from Brute Force Attacks – a trial and error method to find sensitive information like passwords.
41) Verify if system or network is secured from DoS (denial-of-service) attacks. Hacker can target network or single computer with continuous requests due to which resources on target system gets overloaded resulting in denial of service for legit requests.
These are just the basic test scenarios to get started with Pentest. There are hundreds of advanced penetration methods which can be done either manually or with the help of automation tools.
Thursday 23 August 2012
INSTALLATION/UNINSTALLATION TESTING
Have you performed software installation testing? How was the
experience? Well, Installation testing (Implementation Testing) is quite
interesting part of software testing life cycle.
Installation testing is like introducing a guest in your home. The new guest should be properly introduced to all the family members in order to feel him comfortable. Installation of new software is also quite like above example.
If your installation is successful on the new system then customer will be definitely happy but what if things are completely opposite. If installation fails then our program will not work on that system not only this but can leave user’s system badly damaged. User might require to reinstall the full operating system.
In above case will you make any impression on user? Definitely not! Your first impression to make a loyal customer is ruined due to incomplete installation testing. What you need to do for a good first impression? Test the installer appropriately with combination of both manual and automated processes on different machines with different configuration. Major concerned of installation testing is Time! It requires lot of time to even execute a single test case. If you are going to test a big application installer then think about time required to perform such a many test cases on different configurations.
We will see different methods to perform manual installer testing and some basic guideline for automating the installation process.
To start installation testing first decide on how many different system configurations you want to test the installation. Prepare one basic hard disk drive. Format this HDD with most common or default file system, install most common operating system (Windows) on this HDD. Install some basic required components on this HDD. Each time create images of this base HDD and you can create other configurations on this base drive. Make one set of each configuration like Operating system and file format to be used for further testing.
How we can use automation in this process? Well make some systems dedicated for creating basic images (use software’s like Norton Ghost for creating exact images of operating system quickly) of base configuration. This will save your tremendous time in each test case. For example if time to install one OS with basic configuration is say 1 hour then for each test case on fresh OS you will require 1+ hour. But creating image of OS will hardly require 5 to 10 minutes and you will save approximately 40 to 50 minutes!
You can use one operating system with multiple attempts of installation of installer. Each time uninstalling the application and preparing the base state for next test case. Be careful here that your uninstallation program should be tested before and should be working fine.
Installation testing tips with some broad test cases:
1) Use flow diagrams to perform installation testing. Flow diagrams simplify our task. See example flow diagram for basic installation testing test case.
Add some more test cases on this basic flow chart Such as if our application is not the first release then try to add different logical installation paths.
2) If you have previously installed compact basic version of application then in next test case install the full application version on the same path as used for compact version.
3) If you are using flow diagram to test different files to be written on disk while installation then use the same flow diagram in reverse order to test uninstallation of all the installed files on disk.
4) Use flow diagrams to automate the testing efforts. It will be very easy to convert diagrams into automated scripts.
5) Test the installer scripts used for checking the required disk space. If installer is prompting required disk space 1MB, then make sure exactly 1MB is used or whether more disk space utilized during installation. If yes flag this as error.
6) Test disk space requirement on different file system format. Like FAT16 will require more space than efficient NTFS or FAT32 file systems.
7) If possible set a dedicated system for only creating disk images. As said above this will save your testing time.
8 ) Use distributed testing environment in order to carry out installation testing. Distributed environment simply save your time and you can effectively manage all the different test cases from a single machine. The good approach for this is to create a master machine, which will drive different slave machines on network. You can start installation simultaneously on different machine from the master system.
9) Try to automate the routine to test the number of files to be written on disk. You can maintain this file list to be written on disk in and excel sheet and can give this list as a input to automated script that will check each and every path to verify the correct installation.
10) Use software’s available freely in market to verify registry changes on successful installation. Verify the registry changes with your expected change list after installation.
11) Forcefully break the installation process in between. See the behavior of system and whether system recovers to its original state without any issues. You can test this “break of installation” on every installation step.
12) Disk space checking: This is the crucial checking in the installation-testing scenario. You can choose different manual and automated methods to do this checking. In manual methods you can check free disk space available on drive before installation and disk space reported by installer script to check whether installer is calculating and reporting disk space accurately. Check the disk space after the installation to verify accurate usage of installation disk space. Run various combination of disk space availability by using some tools to automatically making disk space full while installation. Check system behavior on low disk space conditions while installation.
13) As you check installation you can test for uninstallation also. Before each new iteration of installation make sure that all the files written to disk are removed after uninstallation. Some times uninstallation routine removes files from only last upgraded installation keeping the old version files untouched. Also check for rebooting option after uninstallation manually and forcefully not to reboot.
I have addressed many areas of manual as well as automated installation testing procedure. Still there are many areas you need to focus on depending on the complexity of your software under installation. These not addressed important tasks includes installation over the network, online installation, patch installation, Database checking on Installation, Shared DLL installation and uninstallation etc.
Installation testing is like introducing a guest in your home. The new guest should be properly introduced to all the family members in order to feel him comfortable. Installation of new software is also quite like above example.
If your installation is successful on the new system then customer will be definitely happy but what if things are completely opposite. If installation fails then our program will not work on that system not only this but can leave user’s system badly damaged. User might require to reinstall the full operating system.
In above case will you make any impression on user? Definitely not! Your first impression to make a loyal customer is ruined due to incomplete installation testing. What you need to do for a good first impression? Test the installer appropriately with combination of both manual and automated processes on different machines with different configuration. Major concerned of installation testing is Time! It requires lot of time to even execute a single test case. If you are going to test a big application installer then think about time required to perform such a many test cases on different configurations.
We will see different methods to perform manual installer testing and some basic guideline for automating the installation process.
To start installation testing first decide on how many different system configurations you want to test the installation. Prepare one basic hard disk drive. Format this HDD with most common or default file system, install most common operating system (Windows) on this HDD. Install some basic required components on this HDD. Each time create images of this base HDD and you can create other configurations on this base drive. Make one set of each configuration like Operating system and file format to be used for further testing.
How we can use automation in this process? Well make some systems dedicated for creating basic images (use software’s like Norton Ghost for creating exact images of operating system quickly) of base configuration. This will save your tremendous time in each test case. For example if time to install one OS with basic configuration is say 1 hour then for each test case on fresh OS you will require 1+ hour. But creating image of OS will hardly require 5 to 10 minutes and you will save approximately 40 to 50 minutes!
You can use one operating system with multiple attempts of installation of installer. Each time uninstalling the application and preparing the base state for next test case. Be careful here that your uninstallation program should be tested before and should be working fine.
Installation testing tips with some broad test cases:
1) Use flow diagrams to perform installation testing. Flow diagrams simplify our task. See example flow diagram for basic installation testing test case.
Add some more test cases on this basic flow chart Such as if our application is not the first release then try to add different logical installation paths.
2) If you have previously installed compact basic version of application then in next test case install the full application version on the same path as used for compact version.
3) If you are using flow diagram to test different files to be written on disk while installation then use the same flow diagram in reverse order to test uninstallation of all the installed files on disk.
4) Use flow diagrams to automate the testing efforts. It will be very easy to convert diagrams into automated scripts.
5) Test the installer scripts used for checking the required disk space. If installer is prompting required disk space 1MB, then make sure exactly 1MB is used or whether more disk space utilized during installation. If yes flag this as error.
6) Test disk space requirement on different file system format. Like FAT16 will require more space than efficient NTFS or FAT32 file systems.
7) If possible set a dedicated system for only creating disk images. As said above this will save your testing time.
8 ) Use distributed testing environment in order to carry out installation testing. Distributed environment simply save your time and you can effectively manage all the different test cases from a single machine. The good approach for this is to create a master machine, which will drive different slave machines on network. You can start installation simultaneously on different machine from the master system.
9) Try to automate the routine to test the number of files to be written on disk. You can maintain this file list to be written on disk in and excel sheet and can give this list as a input to automated script that will check each and every path to verify the correct installation.
10) Use software’s available freely in market to verify registry changes on successful installation. Verify the registry changes with your expected change list after installation.
11) Forcefully break the installation process in between. See the behavior of system and whether system recovers to its original state without any issues. You can test this “break of installation” on every installation step.
12) Disk space checking: This is the crucial checking in the installation-testing scenario. You can choose different manual and automated methods to do this checking. In manual methods you can check free disk space available on drive before installation and disk space reported by installer script to check whether installer is calculating and reporting disk space accurately. Check the disk space after the installation to verify accurate usage of installation disk space. Run various combination of disk space availability by using some tools to automatically making disk space full while installation. Check system behavior on low disk space conditions while installation.
13) As you check installation you can test for uninstallation also. Before each new iteration of installation make sure that all the files written to disk are removed after uninstallation. Some times uninstallation routine removes files from only last upgraded installation keeping the old version files untouched. Also check for rebooting option after uninstallation manually and forcefully not to reboot.
I have addressed many areas of manual as well as automated installation testing procedure. Still there are many areas you need to focus on depending on the complexity of your software under installation. These not addressed important tasks includes installation over the network, online installation, patch installation, Database checking on Installation, Shared DLL installation and uninstallation etc.
Subscribe to:
Posts (Atom)