KaneAI - World’s First E2E Software Testing Agent.Book a Demo 
How to use test_ignores method in prospector
Best Python code snippet using prospector_python
exPscan.py
Source:exPscan.py 
1#!/usr/bin/python22import re3import functools4import traceback5import pickle6import uuid7import pprint8from org.zaproxy.zap.extension.script import ScriptVars9# configuration10DEV_MODE = True11NAME = "exPscan"12MAX_BODY_SIZE = 30000013DATA_TYPES = {14 "js": ["javascript", "ecmascript"],15 "css": ["text/css"],16 "default": None,17 }18# don't touch these globales19_GLOB = {20 "REGEX": dict.fromkeys(DATA_TYPES, ""),21 "IREGEX": dict.fromkeys(DATA_TYPES, ""),22 "REG_BY_IDS": {},23 "ERRORS": "",24 }25_NONREG_STRINGS = """26## vi: ft=conf27## NOTE:28## this file is a aggregate of strings that `should probably` be recognized.29## It is useful for non-regression tests30## * Lines starting with '#' are ignored31##32## vim tips:33## remove duplicates:34## :'<,'>!sort -u35## sort by line length:36## :'<,'>!awk '{ print length, $0 }' | sort -n -s | cut -d" " -f2-37DB2 ODBC38Index of39JDBC SQL40ODBC DB241ODBC SQL42#DB2 Error #(always contains SQLSTATE, or SQL0204N like strings)43PHP Error44 server at 45#CLI Driver # (always contains SQL0420N like str)46#DB2 Driver47JDBC Error48JDBC MySQL49MySQL ODBC50ODBC Error51#Oracle DB2 # useless52Fatal error53JDBC Driver54JDBC Oracle55mysql error56MySQL Error57ODBC Driver58ODBC Oracle59Oracle ODBC60PHP Warning61data source=62Error Report63include_path64Invalid SQL:65MySQL Driver66Oracle Error67SQLException68invalid query69Oracle Driver70Type mismatch71Unknown table72database error73internal error74ODBC SQL Server75PHP Parse error76Parent Directory77unexpected error78ADODB.Field error79#ASP.NET_SessionId # irrelevant80mix of collations81SQL Server Driver82missing expression83server object error84#Warning: pg_connect # already detected to "on line [0-9]" regex in real life85Can't find record in86#Custom Error Message #???87#Warning: mysql_query # already detected ty "on line [0-9]" regex in real life88Incorrect column name89Incorrect syntax near90Internal Server Error91ODBC Microsoft Access92on MySQL result index93The error occurred in94Unable to jump to row95Can't connect to local96Disallowed Parent Path97Invalid parameter type98Invalid Path Character99mySQL error with query100ODBC SQL Server Driver101#Warning: mysql_query()102The script whose uid is103is not allowed to access104#Microsoft VBScript error # already caught in real life by microsoft regex '800a0400'105Microsoft VBScript error '800a0400'106Active Server Pages error107detected an internal error108A syntax error has occurred109Error Diagnostic Information110ODBC Microsoft Access Driver111Unterminated string constant112): encountered SQLException [113SQL Server Driver][SQL Server114unexpected end of SQL command115Permission denied: 'GetObject'116SQL command not properly ended117[ODBC Informix driver][Informix]118OLE/DB provider returned message119Syntax error in query expression120Invalid procedure call or argument121Invision Power Board Database Error122#Microsoft VBScript compilation error # already caught in real life by microsoft regex '800a0400'123You have an error in your SQL syntax124ERROR: parser: parse error at or near125Incorrect column specifier for column126Error Occurred While Processing Request127Microsoft OLE DB Provider for SQL Server128Unexpected end of command in statement [129You have an error in your SQL syntax near130internal error [IBM][CLI Driver][DB2/6000]131Microsoft OLE DB Provider for ODBC Drivers132[Microsoft][ODBC Microsoft Access 97 Driver]133Column count doesn't match value count at row134Error converting data type varchar to numeric135supplied argument is not a valid MySQL result136An unexpected token "END-OF-STATEMENT" was found137Error Message : Error loading required libraries.138java.lang.NumberFormatException: For input string:139Supplied argument is not a valid PostgreSQL result140PostgreSQL query failed: ERROR: parser: parse error141Unclosed quotation mark before the character string142An illegal character has been found in the statement143ASP.NET is configured to show verbose error messages144detected an internal error [IBM][CLI Driver][DB2/6000]145supplied argument is not a valid MySQL result resource146[SQL Server Driver][SQL Server]Line 1: Incorrect syntax near147Warning: Cannot modify header information - headers already sent148Warning: Supplied argument is not a valid File-Handle resource in149Warning: pg_connect(): Unable to connect to PostgreSQL server: FATAL150Incorrect syntax near151query failed152#not an object # too much false positives153error occurred154ERROR OCCURRED155Server Error156invalid file name157fatal error158parse error159ERROR 1049 (42000): Unknown database160No database selected161#exception report # not relevant on google hack search162Servlet error : java.lang.IndexOutOfBoundsException163"""164def exception_handler(function):165 """166 A decorator that wraps the passed in function and outputs167 exception instead if raising it, if DEV_MODE is True168 This is useful to not have to re-enable the script from ZAP169 each time we trigger an exception during development.170 """171 @functools.wraps(function)172 def wrapper(*args, **kwargs):173 if DEV_MODE:174 try:175 return function(*args, **kwargs)176 except:177 print("==== EXCEPTION CATCHED (DEV_MODE) ====")178 print(traceback.format_exc())179 else:180 return function(*args, **kwargs)181 return wrapper182def hash_source_code():183 """184 Get a hash representing the source code of current script185 It remains the same as long as source code has not changed186 """187 import ctypes188 from org.parosproxy.paros.control import Control189 script_name = globals()["zap.script.name"]190 extLoader = Control.getSingleton().getExtensionLoader()191 extScript = extLoader.getExtension("ExtensionScript")192 script_source = extScript.getScript(script_name).getContents()193 h = ctypes.c_uint32(hash(script_source)).value % 0xffffff194 return hex(h)[2:].zfill(6)195def str_to_lines(string):196 """yield non-empty lines from a multi-line string197 """198 for line in string.splitlines():199 if not line.strip():200 continue201 # ignore indentation spaces202 while line[:4] == " ":203 line = line[4:]204 yield line205def sanitize_regex(regex):206 # this will not work anyway with current implementation207 assert not regex.startswith("^")208 assert not regex.endswith("$")209 # make internal groups non-capturing to limit overhead210 assert not "\\\\(" in regex211 regex = regex.replace("\\(", "_-*placeholder1*-_")212 regex = regex.replace("(?:", "_-*placeholder2*-_")213 regex = regex.replace("(", "(?:")214 regex = regex.replace("_-*placeholder1*-_", "\\(")215 regex = regex.replace("_-*placeholder2*-_", "(?:")216 # limit wildcards (.* & .+ can considerably slow down processing time)217 regex = regex.replace(".+", ".{1,40}")218 regex = regex.replace(".*", ".{,40}")219 return regex220def test_fail(obj, regex, line):221 global _GLOB222 word = "IGNORED" if obj else "FOUND"223 out = "-"*50 + "\n"224 out += "[-] Test Failed: line should be %s by regex\n" % word225 if regex:226 out += " REGEX: %s\n" % regex227 if line:228 out += " LINE: %s\n" % line229 if obj:230 out += " MATCH: %r\n\n" % obj231 _GLOB["ERRORS"] += out232def process_regex(raw_regex, issue,233 test_finds="", test_ignores="", flags=0):234 global _GLOB235 issue_id = issue.replace(" ", "_kw_") + str(uuid.uuid4())[:8]236 assert issue_id not in _GLOB["REG_BY_IDS"]237 _GLOB["REG_BY_IDS"][issue_id] = raw_regex238 regex = "(?P<%s>%s)" % (issue_id, sanitize_regex(raw_regex))239 # execute unit tests240 test = re.compile(regex, flags)241 for line in str_to_lines(test_finds):242 res = test.findall("\n"+line+"\n")243 if not res:244 test_fail(res, regex, line)245 for line in str_to_lines(test_ignores):246 res = test.findall("\n"+line+"\n")247 if res:248 test_fail(res, regex, line)249 return regex250def add_strings(issue_name, strings):251 global _GLOB252 for line in str_to_lines(strings):253 regex = process_regex(r"\b%s\b" % line, issue_name)254 for t in DATA_TYPES:255 if _GLOB["REGEX"][t]:256 _GLOB["REGEX"][t] += "|"257 _GLOB["REGEX"][t] += regex258def add_regex(issue_name, regex,259 test_finds, test_ignores="", ignored_types=""):260 global _GLOB261 regex = process_regex(regex, issue_name,262 test_finds, test_ignores)263 ignored_types = ignored_types.split()264 for t in DATA_TYPES:265 if t in ignored_types:266 continue267 if _GLOB["REGEX"][t]:268 _GLOB["REGEX"][t] += "|"269 _GLOB["REGEX"][t] += regex270def add_iregex(issue_name, regex,271 test_finds, test_ignores="", ignored_types=""):272 global _GLOB273 regex = process_regex(regex, issue_name,274 test_finds, test_ignores, re.I)275 ignored_types = ignored_types.split()276 for t in DATA_TYPES:277 if t in ignored_types:278 continue279 if _GLOB["IREGEX"][t]:280 _GLOB["IREGEX"][t] += "|"281 _GLOB["IREGEX"][t] += regex282def build_matcher():283 ############################################################284 name = "PHP Source code disclosure"285 add_regex(name, r"<\?(php\s|\=)",286 test_finds = """287 data="<?php288 <?=$data;?>289 """,290 test_ignores = """291 <?PhP292 <?PhPa293 <?PhP0294 <?295 < ? php296 < ? =297 """)298 add_strings(name, " => Array")299 add_regex(name, r"\$[a-zA-Z_][a-zA-Z0-9_]+\[",300 test_finds = """301 mysqli_connect($config['host'], 302 $_POST[0]303 $_GET["x"]304 $ee[305 """,306 test_ignores = """307 $#[308 $1[309 $$_GET ["x"]310 $_GET ["x"]311 a$a[312 $e[313 """,314 ignored_types = "js")315 ############################################################316 name = "JAVA Source code disclosure"317 add_regex(name, r'\bimport javax?\.[a-zA-Z0-9.]+;',318 test_finds = """319 import java.io.File;320 import java.net.MalformedURLException;321 import javax.servlet.http.HttpServlet;322 """,323 test_ignores = """324 Ximport javax.servlet.http.HttpServlet;325 """)326 add_regex(name, r'\bclass( \w+){1,3}\s*\{',327 test_finds = """328 public class SimpleServlet extends HttpServlet {329 public class TestGate {330 public class TestGate{331 """,332 test_ignores = """333 public class {334 """)335 ############################################################336 name = "ASP Source code disclosure"337 add_strings(name, "On Error Resume Next")338 ############################################################339 name = "ASP NET Source code disclosure"340 add_regex(name, r'@Render[A-Z][a-z]+',341 test_finds = """342 @RenderPage343 @RenderBody344 @RenderSection345 """)346 ############################################################347 name = "C Source code disclosure"348 add_regex(name, r'#(include|define|ifn?def|endif)\b',349 test_finds = """350 #include x351 #define352 #ifdef353 #ifndef354 #endif355 """,356 test_ignores = """357 #includes358 """)359 ############################################################360 name = "Cold Fusion Source code disclosure"361 add_regex(name, r'<cf(argument|component|dump|else|elseif|execute|exit|function|if|loop|output|query|queryparam|return|script|set)\b',362 test_finds = """363 <cfargument364 <cfcomponent365 <cfdump366 <cfelse367 <cfelseif368 <cfexecute369 <cfexit370 <cffunction371 <cfif372 <cfloop373 <cfloop374 <cfoutput375 <cfquery376 <cfqueryparam377 <cfreturn378 <cfscript379 <cfset380 """,381 test_ignores = """382 <cfX383 <cfx384 <cf385 <CFIF386 <CfDump387 """)388 ############################################################389 name = "Source code disclosure"390 add_regex(name, r'[A-Za-z._]+(Exception|Error|Controller|Servlet|Object|Client|Connection|Driver)([^a-z]|$)',391 test_finds = """392 System.Exception393 SQLException394 SQLite/JDBCDriver395 AppDomain.CurrentDomain.UnhandledException396 java.lang.RuntimeException397 Type: RuntimeException398 aspController()399 MysqlController.CheckMysqlIsRunning()400 ErrorController.php5</b> on line <b>73</b><br />401 AuthPluginController.php on line <i>58</i>402 RuntimeError: Expected object of type403 (RuntimeError) Element does not exist in cache404 @WebServlet405 HTTPServlet406 Server.CreateObject 407 of type 'System.__ComObject408 The type or namespace name `Data.MySqlClient' could not be found.409 Class 'mysqlConnection' not found.410 Zend_Db_Statement_Db2_Exception411 Zend_Db_Adapter_Db2_Exception412 ArrayObject Object413 Servlet error : java.lang.IndexOutOfBoundsException414 """,415 test_ignores = """416 Exception417 XExceptions418 Errors419 Bad Error.420 Controllers421 #Controller422 """,423 ignored_types = "js css")424 add_regex(name, r' runat=',425 test_finds = """426 <umbraco:Macro runat="server" language="cshtml">427 <script runat="server">428 <asp:Foo runat="server">429 <head id="Head1" runat="server">430 """)431 add_regex(name, r"<%(@|=)?\s*[A-Za-z]{2,}",432 test_finds = """433 <%@ taglib prefix="jcr"434 <%@ include file="webftpclass.jsp"435 <%@ page errorPage436 <%@ Page Language="C#" %>437 <% int x = 5; %>438 <%@ Page Inherits="ParentPageClass" %>439 <% Sub SomeProcedure() %>440 <% End Sub %>441 <%Assembly442 <%OutputCache443 <%Implements444 """)445 add_regex(name, r'\b(static\s+void|void\s+static)\b',446 test_finds = """447 public void static main(448 public static void main(449 public static void main(450 """)451 add_regex(name, r"<[aj]sp:[a-zA-Z]",452 test_finds = """453 <jsp:directive.taglib uri = "uri" prefix = "prefixOfTag" />454 <asp:TreeView id="SiteTreeView" DataSourceID=455 <asp:SiteMapDataSource id="MenuSource"456 """)457 add_iregex(name, r'\s@(for\s?each|switch|select|interface|implementation|protocol|private|synthesize|property)\s',458 test_finds = """459 @interface Foo : NSObject {460 @PRIVATE461 @foreach462 @For Each463 @property464 @synthesize465 """,466 test_ignores = """467 @For EachX468 @foreachX469 @ends470 admin@private.com471 admin@privates.com472 """)473 add_iregex(name, r'\b(connection|query)string\b',474 test_finds = """475 ConnectionString = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & m_strDatabase & ";Persist Security Info=False"476 Convert.ToInt32(Request.QueryString["pID"]);477 oledb.oledbConnection(connectionString)'478 oRequest.querystring 479 """,480 test_ignores = """481 Parse and stringify URL query strings482 A query string is the portion of ...483 connection string484 """,485 ignored_types = "js")486 add_iregex(name, r'\bdata\s*source\s*=',487 test_finds = """488 ConnectionString = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & m_strDatabase & ";Persist Security Info=False"489 Data Source={DataDirectory}\test.db;490 "Data Source=(local);Database=Northwind;User ID=springqa;Password=springqa;491 """)492 ############################################################493 name = "ASP Error message"494 add_regex(name, r"\bASP [01]\d{3}\b",495 test_finds = """496 ctive Server Pages error 'ASP 0131', Disallowed Parent497 Error Message: ASP 0131, Disallowed Parent 498 Active Server Pages error 'ASP 0131'499 Active Server Pages error 'ASP 0113' 500 'ASP 0115' Unexpected Error501 """,502 test_ignores = """503 ASP 2008504 ASP 2000505 """)506 add_strings(name, """507 Active Server Pages error508 Disallowed Parent Path509 """)510 ############################################################511 name = "PHP Error message"512 add_regex(name, r"<b>(Notice|Deprecated)</b>: ",513 test_finds = """514 <b>Notice</b>: Undefined offset: 6515 <b>Deprecated</b>: Non-static method516 """,517 test_ignores = """518 """)519 add_iregex(name, r"\b(warning|error)\b.*?: +(<.+?>)?[a-zA-Z_][a-zA-Z0-9_]+\(",520 test_finds = """521 <b>Warning</b>: require_once(../body.asp)522 ;;Warning: require_once(../body.asp)523 /Warning: require_once(../body.asp)524 Warning: require_once(../body.asp)525 <b>Fatal error</b>: require_once() 526 <b>Warning</b> (2)</a>: mysqli_connect() 527 Error: </td><td colspan='2'>mysql_connect(): Lost connection to MySQL server528 <b>Warning</b>: mysqli_fetch_assoc()529 """,530 test_ignores = """531 error: x()532 fatal error: require_once ()533 """,534 ignored_types = "js")535 add_regex(name, r'</b> on line <b>[0-9]+</b>',536 test_finds = """537 in <b>D:\wwwroot\test.asp</b> on line <b>2</b>538 ErrorController.php5</b> on line <b>73</b><br />539 errorhandler.php</b> on line <b>218</b><br>540 wp-config.php.old</b> on line <b>61</b><br />541 /dvds.cfm</b> on line <b>166</b><br>542 main_index.cgi</b> on line <b>75</b><br>543 WebCalendar.class</b> on line <b>18</b>544 """,545 test_ignores = """546 """)547 # [10-May-2014 04:58:13 UTC] PHP Warning: Module 'newrelic' already loaded in Unknown on line 0548 # Warning: Cannot modify header information - headers already sent549 # Warning: Supplied argument is not a valid File-Handle resource in550 add_strings(name, """551 PHP Warning552 Cannot modify header information553 Supplied argument is not a valid554 include_path555 The script whose uid is556 """)557 ############################################################558 name = "Error message"559 add_regex(name, r"\.(aspx?|php[0-9]?|inc|cfm|old|cgi|jsp|html?|class)\b.*?\b[Ll]ine\b.*?[0-9]+",560 test_finds = """561 /download.asp</font><font face="Arial" size=2>, line 12</font>562 /download.asp, line 12563 route.php, line 12564 route.inc, line 12565 APP/webroot/index.php, line 87</pre>566 C:\html\pdf.aspx<b> Line: </b> 334567 \SQLServer_connection.aspx Line: 33 568 events.aspx:line 14 -->569 AuthPluginController.php on line <i>58</i>570 should not be called statically in file /home/storage/handler.php line 230<br>571 Error in C:\inetpub\Default.aspx on line 322572 /new.php on line 2573 """,574 test_ignores = """575 asp, line576 aspon line577 xasp, line578 Xphp, lineXX3X579 XPHPXLINE580 """,581 ignored_types="css")582 add_regex(name, r"\.(aspx?|php[0-9]?|inc|cfm|old|cgi|jsp|html?|class)\b\s*(<.*?>)?\s*:\s*(<.*?>)?\s*[0-9]+",583 test_finds = """584 in e:\WWW\pdf.aspx:380585 (output started at /wwwroot/err.php:272)586 in /home/storage/2/21/af/sbmp/public_html/cbab/siscbab/include/common.php:63587 >../Bootstrap.php<b>:</b>97</td></tr>588 undefined function mysql_connect() in /home1/dbi.php:105589 """,590 test_ignores = """591 """)592 add_iregex(name, r"\bbacktrace\b",593 test_finds = """594 X backTrace X595 class.backTrace596 """,597 test_ignores = """598 backtraces599 xbackTrace600 """)601 add_iregex(name, r"\bstack.trace\b",602 test_finds = """603 <b>Stack Trace:</b> <br><br>604 Please review the stack trace for more information605 stack-trace606 """,607 test_ignores = """608 """,609 ignored_types = "js")610 add_iregex(name, r"\bunable to cast ",611 test_finds = """612 Unable to cast object of type 613 Unable to cast object of type 'proje.hesap' to type614 Unable to cast COM object of type 'System.__ComObject'615 """,616 test_ignores = """617 """)618 add_iregex(name, r"\b(internal|fatal|unhandled|unexpected|uncaught) ?(exception|error)\b",619 test_finds = """620 An unexpected error occurred on a send.621 unhandled error622 internal error623 Connection Lost: Internal Exception: java.io.IOException: An established connection was aborted624 """,625 test_ignores = """626 Xinternal error627 """,628 ignored_types = "js")629 add_iregex(name, r"\b(syntax|parse|runtime) error\b",630 test_finds = """631 syntax error near unexpected token `('632 Error: syntax error near \":=\" : expected '.'633 org.postgresql.util.PSQLException: ERROR: syntax error at or near "$1"634 Error :: ERROR: syntax error at or near "TYPE" at character 51635 ERROR: parser: parse error at or near636 Microsoft VBScript runtime error '800a000d' Type mismatch637 """,638 test_ignores = """639 runtime X error640 runtime errors641 """,642 ignored_types = "js")643 add_iregex(name, r"\b(error|exception)\b.*?\bwhile (attempting|trying) to ",644 test_finds = """645 ERROR: I/O or zip error while attempting to read entry646 Error while attempting to commit transaction.647 There was an unexpected error while trying to repair Trusted648 Error while trying to run project : Unable to start debugging.649 NamingException: Exception while trying to get InitialContext. 650 threw an exception while trying to deserialize the 651 WARNING: Exception while attempting to add an entry652 [CASSANDRA-12152] Unknown exception caught while attempting to update653 Exception thrown while attempting to traverse the result set [654 An unexpected exception occurred while attempting to communicate655 MODx encountered the following error while attempting to '656 """,657 ignored_types = "js")658 add_iregex(name, r"\b(error|exception) (\w+ )?(encountered|occurr?ed)\b",659 test_finds = """660 A PHP Error was encountered661 <h4>A PHP Error was encountered</h4>662 Warning. Error encountered while saving cache663 Exception encountered during initialization664 An unexpected exception occurred while attempting to communicate665 A COM exception has occured.666 """,667 test_ignores = """668 Xerrror was encountered669 """,670 ignored_types = "js")671 add_iregex(name, r"\error (occurr?ed|loading|encountered|report|message|converting|diagnostic)",672 test_finds = """673 error messages674 error loadinG675 Error Diagnostic Information676 Error Report677 """,678 ignored_types = "js")679 add_regex(name, r'\b[Ee]ncountered an? (\w+ )?(error|exception)\b',680 test_finds = """681 BackupManager encountered an exception682 Sorry, we encountered an error....683 TypeError: ServiceWorker script encountered an error during684 NTVDM encountered a hard error685 [Server thread/ERROR]: Encountered an unexpected exception net.686 encountered a declaration exception687 Debugger encountered an exception: Exception at 0x7ffd21349e08688 DJLDAPv3Repo encountered an ldap exception.689 The server encountered a temporary error and could not complete your request690 """,691 test_ignores = """692 But I encountered with a strange error, it says,693 If the error encountered is a softer error, such as an ...694 """)695 add_iregex(name, r'\bconnection (was |is )?closed\b',696 test_finds = """697 The underlying connection was closed698 An unrecoverable IOException occurred so the connection was closed.699 System.Net.WebException: The underlying connection was closed700 Database connection closed on port701 Connection closed by foreign host.702 Exception:Message: The connection is closed703 """,704 test_ignores = """705 Xconnection was closed706 connection was closedX707 """)708 add_iregex(name, r'\b(php|server) (\w+ )?error\b',709 test_finds = """710 « PHP Parse Error »</b>711 <b>PHP error debug</b>712 <H1>Server Error in '/EMCFLEXQUOTE' Application.<hr713 """,714 test_ignores = """715 some php errors716 observer error717 servererror718 server errors719 """,720 ignored_types = "js")721 add_iregex(name, r'(login|access|authentication|permission) (failed|failure|denied)',722 test_finds = """723 Authentication failed for user root@localhost.localdomain.724 FATAL: Ident authentication failed for user "pgadmin" 725 Access denied for user 726 Microsoft VBScript runtime (0x800A0046). Permission denied.727 SQLSTATE[HY000] [1045] Access denied for user 'root'@'localhost'728 """,729 ignored_types = "js")730 add_iregex(name, r'\b(unterminated|unexpected) (end|token|string)\b',731 test_finds = """732 Parse error: syntax error, unexpected end of file in733 compile error: unexpected end of script734 unexpected end-of-file occurred735 ORA-00921: unexpected end of SQL command736 syntax error near unexpected token `('737 An unexpected token "END-OF-STATEMENT" was found738 Unterminated string constant 739 SyntaxError: unterminated string literal740 error: unterminated string741 """,742 ignored_types = "js")743 add_strings(name, """744 is not allowed to access745 """)746 ############################################################747 name = "Microsoft Error message"748 add_iregex(name, r'[x\W\b]800(40|a0|04)[a-f0-9]{3}\b',749 test_finds = """750 Microsoft VBScript runtime error '800a000d' Type mismatch751 6 Microsoft JET Database Engine Error '80004005' 752 Microsoft SQL Native Client error '80004005' Named Pipes753 Microsoft VBScript runtime (0x800A0046). Permission denied.754 Microsoft SQL Native Client error '80004005'. Cannot open database755 Microsoft SQL Native Client error '80004005'. Login failed for user 'Admin'756 Microsoft SQL Native Client error '80040e37'.757 """,758 test_ignores = """759 """)760 ############################################################761 name = "DB2 SQL Error message"762 add_regex(name, r"\bSQL\d{4}N\b",763 test_finds = """764 [IBM][CLI Driver][DB2] SQL0443N Routine "SQLTABLES" (specific name "SQLTABLES") has returned an error SQLSTATE with765 During SQL processing it returned: SQL0204N766 nput Data (33) Error SQLExtendedFetch: [IBM][CLI Driver][DB2/AIX64] SQL0420N Invalid character found in a character string argument of the function "DECFLOAT". SQLSTATE=22018767 """)768 ############################################################769 name = "Oracle SQL Error message"770 add_regex(name, r"(\b|^)ORA-\d{5}(\b|$)",771 test_finds = """772 ORA-00921: unexpected end of SQL command773 ORA-29282: invalid file ID774 ORA-00933: SQL command not properly terminated. 775 ou,ORA-28002:,the,password,776 """)777 ############################################################778 name = "SQL Error message"779 #check the manual that corresponds to your MySQL server version for780 #check the manual that corresponds to your MariaDB server version for781 #error in your SQL syntax782 add_strings(name, """783 error in your SQL syntax784 check the manual that corresponds to your785 Can't find record in786 Type mismatch787 mix of collations788 Unable to jump to row789 missing expression790 Can't connect to local791 Invalid Path Character792 Column count doesn't match value count at row793 Unclosed quotation mark before the character string794 An illegal character has been found in the statement795 No database selected796 """)797 add_iregex(name, r"\b(unknown|invalid|incorrect) (column|table|query|sql|parameter|procedure|syntax|database|file)",798 test_finds = """799 Error 1054 Unknown column 'a.category' in 'where clause'800 Debug info: Unknown column 'groupmembersonly' in 'where clause' SELECT801 Unknown column 'a.id' in 'on clause' 802 PHP Fatal error: 1054 :Unknown column 'status' in 'where clause'803 Data.SqlClient.SqlException: Invalid column name 'apikeytime'.804 [Microsoft][SQL Server Native Client 10.0][SQL Server]Invalid column name 'U_FOC'.805 Warning: Requested unknown parameter806 stored procedure: Invalid Parameter Type.807 Invalid procedure call or argument808 Incorrect syntax near809 ORA-29282: invalid file ID810 """,811 ignored_types = "js")812 add_iregex(name, r"sql[ _]?(state|code)\b",813 test_finds = """814 Connection failed : SQL state S1 T00 SQL SERVER ERROR 0815 SQL State 3000816 SQL STATE: S1000 ERROR CODE: -25. ERROR IN SCRIPT LINE FILE: 78'817 Database error: SQL State 'HYC00';818 [SQLDriverConnect]{SQL_STATE: IM002}[Microsoft][ODBC Driver Manager]'819 Warning C4251 'sql::SQLException::sql_state': class 'std::820 SQLSTATE[HY000] [14] unable to open database file821 SQLSTATE[HY000] [1045] Access denied for user 'root'@'localhost'822 SQLSTATE[42000]: Syntax error or access violation: 1064823 mysql state: 28000824 [IBM][CLI Driver][DB2] SQL0443N Routine "SQLTABLES" (specific name "SQLTABLES") has returned an error SQLSTATE with825 Exception: [Informix][Informix ODBC Driver]Driver not capable. SQLCODE=-11092826 nput Data (33) Error SQLExtendedFetch: [IBM][CLI Driver][DB2/AIX64] SQL0420N Invalid character found in a character string argument of the function "DECFLOAT". SQLSTATE=22018827 """,828 test_ignores = """829 sqlstates830 sql states831 sql codes832 """)833 add_regex(name, r"\b not properly (ended|terminated)\b",834 test_finds = """835 quoted string not properly terminated836 SQL command not properly ended837 ORA-00933: SQL command not properly terminated. 838 """,839 test_ignores = """840 """)841 add_iregex(name, r"\b[jo]dbc\b",842 test_finds = """843 com.mysql.jdbc844 org.postgresql.jdbc845 SQLServer JDBC Driver846 macromedia.jdbc.sqlserver847 com.microsoft.sqlserver.jdbc848 macromedia.jdbc.oracle849 oracle.jdbc850 com.informix.jdbc851 weblogic.jdbc.informix852 org.firebirdsql.jdbc853 org.sqlite.JDBC854 com.sap.dbtech.jdbc855 com.sybase.jdbc856 com.ingres.gcf.jdbc857 com.frontbase.jdbc858 org.hsqldb.jdbc859 org.h2.jdbc860 com.microsoft.sqlserver.jdbc.SQLServerException: Violation of PRIMARY KEY constraint861 .jdbc'862 Exception: [Informix][Informix ODBC Driver]Driver not capable. SQLCODE=-11092863 [Microsoft][ODBC SQL Server Driver][SQL Server]864 PostgreSQL ODBC error865 [SQLDriverConnect]{SQL_STATE: IM002}[Microsoft][ODBC Driver Manager]'866 Warning: SQL error: [INTERSOLV][ODBC SQL Server driver][SQL Server]Invalid column name 'projanfang'867 [S1090] [Microsoft][ODBC DB2 Driver] Invalid string or buffer length.868 """,869 test_ignores = """870 xjdbc871 jdbcx872 """)873 add_iregex(name, r'(sql|\bdb2|\b[oj]dbc|\bsqlite|\bdatabase|\boracle|\bdb|\bquery)\s+(\w+\s+)?(driver|failed|error|exception|warning|engine)\b',874 test_finds = """875 Access Database Engine876 <b>Database error:</b> Invalid SQL:877 <b>MySQL Error</b>: 878 Database error occured: #1950 (2627) Generic db error: "2627 879 Database error: SQL State 'HYC00';880 Dynamic SQL Error SQL Error code = -204 Table UNKNOW881 Exception: [Informix][Informix ODBC Driver]Driver not capable. SQLCODE=-11092882 JET Database Engine883 >« Execution of a query to the database failed »884 [Microsoft][ODBC SQL Server Driver][SQL Server]885 Postgresql CDC Error886 PostgreSQL ODBC error887 PostgreSQL query failed888 [SQLDriverConnect]{SQL_STATE: IM002}[Microsoft][ODBC Driver Manager]889 SQLite error 11: database disk 890 The Microsoft Jet database engine cannot open the file891 Warning: SQL error: [INTERSOLV][ODBC SQL Server driver][SQL Server]Invalid column name 'projanfang'892 """,893 test_ignores = """894 some sql errors895 """)896 add_regex(name, r'(oledb|OLE[\W_]?DB)\b',897 test_finds = """898 oraOLEDB.Oracle: Provider not found: Oracle 1899 provider=MYSQLOLEDB; Driver={MySQL};SERVER=localhost900 Codigo de Excepcion OLE IDispatch 0 de Microsoft OLE DB Provider SQL Server901 oledb.oledbConnection(connectionString)'902 OraOLEDB903 Provider=Microsoft.Jet.OLEDB.4.0;904 OLE_DB905 """,906 test_ignores = """907 OLE_DBX908 XOLE_DBX909 OL_DBX910 """)911 # removed 'ingres' because of false positives912 add_iregex(name, r'(^|[\W\s_])(adodb|informix|sybase|sqlite|mssql|mysql|oracle|hsqldb)([\W\s_]|$)',913 test_finds = """914 X_INFORMIX_X915 com.informix.jdbc916 weblogic.jdbc.informix917 Exception: [Informix][Informix ODBC Driver]Driver not capable. SQLCODE=-11092918 [ADODB.Connection]919 """,920 test_ignores = """921 sybaseX922 Xsybase923 """)924 ############################################################925 name = "SQL Query disclosure"926 add_iregex(name, r'\b(create|drop)\s+(database|table|view|index|procedure|function)\b',927 test_finds = """928 create Database929 create table930 CREATE view931 creAte index932 create procedure933 create\tfunction934 drop Database935 drop table936 DROP view937 drop index938 drop procedure939 drop function940 """,941 test_ignores = """942 create\tfunctions943 Xdrop Database944 """)945 ############################################################946 name = "Leaked file path"947 add_strings(name, "file://")948 add_regex(name, r'(/|\\)Users(/|\\)',949 test_finds = """950 marketplace_file:///C:/Users/sh951 """,952 test_ignores = """953 inetpubX954 """)955 add_regex(name, r'\b[CDE]:(/|\\)',956 test_finds = r"""957 /C:/bla958 E:\bla959 """,960 test_ignores = """961 C:962 XC:/bla963 """)964 add_regex(name, r"/(home|var|www|usr)/",965 test_finds = """966 /home/www/967 daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin968 """,969 test_ignores = """970 homepage971 /HOME/972 # www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449973 """)974 add_regex(name, r"\b(public_html|wwwroot|inetpub|xampp|htdocs)\b",975 test_finds = """976 /var/www/public_html/index.php977 on wwwroot978 in c:/inetpub/wwwroot979 in D:\Inetpub\wwwroot980 """,981 test_ignores = """982 wwwroots983 Public_Html984 """)985 ############################################################986 name = "Unix Shebang disclosure"987 add_regex(name, r"[\s^]#!/(bin|usr)/",988 test_finds = """989 #!/usr/bin/perl990 #!/bin/bash991 """,992 test_ignores = """993 $#!/usr/bin/perl994 "#!/usr/bin/perl"995 """)996 ############################################################997 name = "Directory Listing"998 add_regex(name, r'\b(Index of|Parent Directory|Directory [Ll]isting)\b',999 test_finds = """1000 <title>Directory Listing For /</title>1001 <H1>Directory listing for /</H1>1002 <title>Directory Listing</title>1003 <h1>Index of /phy1004 >Parent Directory</a>1005 Parent Directory1006 <h1>Index of /p1007 """,1008 ignored_types= "js")1009 add_regex(name, r' alt="\[(DIR|PARENTDIR|IMG)\]"',1010 test_finds = """1011 <td style="padding-right:15px"><img src="/layout/i/folder.gif" alt="[DIR]"></td>1012 <img src="/__ovh_icons/back.gif" alt="[PARENTDIR]"> <a href="/content/">Parent Directory</a> 1013 <img src="/__ovh_icons/image2.gif" alt="[IMG]"> 1014 """)1015 ############################################################1016 name = "Product disclosure"1017 add_iregex(name, r"\bpowered by\b",1018 test_finds = """1019 Powered by: vBulletin v3.8.41020 """,1021 test_ignores = """1022 """)1023 add_iregex(name, r" server at ",1024 test_finds = """1025 Proudly Served by LiteSpeed Web Server at xxx.com1026 """)1027 ############################################################1028 name = "Google Analytics UA Tracking ID"1029 # https://stackoverflow.com/questions/2497294/regular-expression-to-validate-a-google-analytics-ua-number1030 # https://www.drupal.org/project/google_analytics/issues/13362521031 add_regex(name, r"\WUA-\d{4,10}(-\d{1,4})?[^\w-]",1032 test_finds = """1033 "UA-12345678"1034 "UA-12345678-12"1035 """,1036 test_ignores = """1037 XUA-12345678"1038 "UA-12345678-12X"1039 "UA-12345678X"1040 """)1041 ############################################################1042 name = "E Mail disclosure"1043 add_regex(name, r"(?:\bmailto:)[^'\" ]+",1044 test_finds = """1045 maintained by <A HREF="mailto:lonsa@ncu.edu">1046 """)1047 add_regex(name, r"\b[\w.+-]+@[a-zA-Z0-9]+[a-zA-Z0-9-]*\.[a-zA-Z0-9-.]*[a-zA-Z0-9]{2,}",1048 test_finds = """1049 "admin@test.com"1050 """,1051 test_ignores = """1052 *@1231053 $@#!.1054 """)1055 # build `matcher` (regex_list, regex_ids)1056 regex_list = {"REGEX": {}, "IREGEX": {}}1057 for dt in DATA_TYPES.keys():1058 regex_list["REGEX"][dt] = _GLOB["REGEX"][dt]1059 regex_list["IREGEX"][dt] = _GLOB["IREGEX"][dt]1060 regex_ids = _GLOB["REG_BY_IDS"]1061 matcher = (regex_list, regex_ids)1062 # nonreg test: ensure all lines are matched as suspicious1063 for line in str_to_lines(_NONREG_STRINGS):1064 if line.startswith("#"):1065 continue1066 res = scan_body(line, "default", matcher)1067 if not res:1068 test_fail(res, None, line)1069 1070 # display errors if any test failed1071 if _GLOB["ERRORS"]:1072 print(_GLOB["ERRORS"])1073 raise Exception("Some tests failed")1074 if DEV_MODE:1075 print("[+] build_matcher(): Matcher successfully built")1076 return matcher1077def scan_body(data, data_type, matcher):1078 regex_list, regex_ids = matcher1079 matches = {}1080 for regex_type in regex_list.keys():1081 regex = regex_list[regex_type][data_type]1082 flags = re.I if regex_type == "IREGEX" else 01083 regex = re.compile(regex, flags)1084 for m in regex.finditer(data):1085 start_pos = m.start()1086 issue_id = m.lastgroup1087 matches[start_pos] = {1088 "str": m.group(issue_id),1089 "regex": regex_ids[issue_id],1090 "issue": issue_id[:-8].replace("_kw_", " "),1091 }1092 return matches1093def get_data_type(content_type):1094 """get the data type (one of DATA_TYPES keys)1095 """1096 for key, val in DATA_TYPES.items():1097 if key == "default":1098 continue1099 for match in val:1100 if match in content_type:1101 return key1102 return "default"1103@exception_handler1104def scan(ps, msg, src):1105 if DEV_MODE:1106 print("\n--------------------")1107 print("[*] %s script started" % NAME)1108 # Docs on alert raising function:1109 # raiseAlert(int risk, int confidence, str name, str description, str uri,1110 # str param, str attack, str otherInfo, str solution,1111 # str evidence, int cweId, int wascId, HttpMessage msg)1112 # risk: 0: info, 1: low, 2: medium, 3: high1113 # confidence: 0: falsePositive, 1: low, 2: medium, 3: high, 4: confirmed1114 old_script_hash = max(ScriptVars.getGlobalVar(NAME+"_hash"), "")1115 script_hash = hash_source_code()1116 if script_hash == old_script_hash:1117 matcher = pickle.loads(ScriptVars.getGlobalVar(NAME+"_matcher"))1118 else:1119 ScriptVars.setGlobalVar(NAME+"_hash", script_hash)1120 matcher = build_matcher()1121 ScriptVars.setGlobalVar(NAME+"_matcher", pickle.dumps(matcher))1122 if DEV_MODE:1123 print("[+] Got matcher, now scanning body")1124 body = msg.getResponseBody()1125 hdr = msg.getResponseHeader()1126 uri = msg.getRequestHeader().getURI().toString()1127 if DEV_MODE:1128 print("[*] URI = %s" % uri)1129 content_type = max(hdr.getHeader(hdr.CONTENT_TYPE), "")1130 content_type = content_type.split(";", 1)[0].strip()1131 blacklist = ["audio/", "video/"]1132 if any(s in content_type for s in blacklist):1133 if DEV_MODE:1134 print("[-] Blacklisted content-type %r: aborting" % content_type)1135 return1136 data = body.toString()[:MAX_BODY_SIZE]1137 data_type = get_data_type(content_type)1138 if DEV_MODE:1139 print("[*] data_type = %s" % data_type)1140 matches = scan_body(data, data_type, matcher)1141 found_evidences = []1142 for start_pos in sorted(matches):1143 match = matches[start_pos]1144 title = "%s: %s (script)" % (NAME, match["issue"])1145 desc = "Regular Expression:\n %s" % match["regex"]1146 evidence = match["str"]1147 if evidence in found_evidences:1148 continue1149 found_evidences.append(evidence)1150 if DEV_MODE:1151 print(" -> GOT MATCH: %s" % title)1152 ps.raiseAlert(0, 1, title, desc, uri, None,1153 None, None, None, evidence, 0, 0, msg)1154 if DEV_MODE:1155 print("[+] Body correctly scanned")1156def appliesToHistoryType(histType):1157 """1158 Limit scanned history types, which otherwise default to1159 types in `PluginPassiveScanner.getDefaultHistoryTypes()`1160 """1161 #from org.parosproxy.paros.model import HistoryReference as hr1162 from org.zaproxy.zap.extension.pscan import PluginPassiveScanner1163 #return histType in [hr.TYPE_PROXIED, hr.TYPE_SPIDER]...
test_helpers.py
Source:test_helpers.py
1import dill2import codecs3import __main__ as _main_module4from nose.tools import with_setup, assert_equal, assert_in, assert_not_in, assert_true, assert_false5from convect.tests import setup_newsgroups_func, teardown_newsgroups_func, setup_ipython_module_func, teardown_ipython_module_func6from convect.helpers import is_iterable, list_large_vars, exclude_ignores_and_redump7def test_is_iterable():8 assert_true(is_iterable(['a', 'b', 'c']))9 assert_true(is_iterable(('a', 'b', 'c',)))10 assert_false(is_iterable('abc'))11def test_list_large_vars_zero_large_vars():12 pickled_session = codecs.encode(13 dill.dumps(_main_module), "base64_codec"14 ).decode()15 assert_equal(list_large_vars(pickled_session), [])16@with_setup(setup_newsgroups_func, teardown_newsgroups_func)17def test_list_large_vars_one_large_var():18 pickled_session = codecs.encode(19 dill.dumps(_main_module), "base64_codec"20 ).decode()21 assert_in('newsgroups', list_large_vars(pickled_session))22@with_setup(setup_ipython_module_func, teardown_ipython_module_func)23def test_list_large_vars_skip_ipython_var():24 assert_true(hasattr(_main_module, "ipython_module"))25 pickled_module = codecs.encode(26 dill.dumps(_main_module), "base64_codec"27 ).decode()28 assert_not_in('ipython_module', list_large_vars(pickled_module))29def test_exclude_ignores_and_redump():30 setattr(_main_module, 'test_ignores', 1)31 assert_in('test_ignores', dir(_main_module))32 assert_equal(_main_module.test_ignores, 1)33 pickled_session = codecs.encode(34 dill.dumps(_main_module), "base64_codec"35 ).decode()36 new_pickled_session = exclude_ignores_and_redump(37 pickled_session, ['test_ignores']38 )39 new_session = dill.loads(40 codecs.decode(41 new_pickled_session.encode(), "base64_codec"42 )43 )...
check_and_format.py
Source:check_and_format.py
1#!/usr/bin/env python32# Copyright (c) 2020 Graphcore Ltd. All rights reserved.3import argparse4import subprocess5import sys6def sh(*cmd): # pylint: disable=invalid-name7 """Run a shell command, terminating on failure"""8 exitcode = subprocess.call(" ".join(map(str, cmd)), shell=True)9 if exitcode:10 sys.exit(exitcode)11parser = argparse.ArgumentParser(description=__doc__)12parser.add_argument(13 "--ci", action="store_true", help="Run in CI mode (check don't autoformat)"14)15args = parser.parse_args()16package = "." # pylint: disable=invalid-name17# Autoformat18sh(f"black {package}/", "--check" if args.ci else "")19# Lint20sh(f"flake8 {package}/")21sh(f"python -m pylint {package} --ignore {package}/tests")22test_ignores = [23 "missing-module-docstring",24 "missing-function-docstring",25 "missing-class-docstring",26 "redefined-outer-name",27 "unused-argument",28 "protected-access",29 "blacklisted-name",30]31sh(32 f"python -m pylint {package}/tests --ignore test_mixins.py",33 *[f"-d {ignore}" for ignore in test_ignores],34)35# Tests...
Automation Testing Tutorials
Learn to execute automation testing from scratch with LambdaTest Learning Hub. Right from setting up the prerequisites to run your first automation test, to following best practices and diving deeper into advanced test scenarios. LambdaTest Learning Hubs compile a list of step-by-step guides to help you be proficient with different test automation frameworks i.e. Selenium, Cypress, TestNG etc.
LambdaTest Learning Hubs:
- JUnit Tutorial
- TestNG Tutorial
- Webdriver Tutorial
- WebDriverIO Tutorial
- Protractor Tutorial
- Selenium 4 Tutorial
- Jenkins Tutorial
- NUnit Tutorial
- Jest Tutorial
- Playwright Tutorial
- Cypress Tutorial
- PyTest Tutorial
YouTube
You could also refer to video tutorials over LambdaTest YouTube channel to get step by step demonstration from industry experts.
Run prospector automation tests on LambdaTest cloud grid
Perform automation testing on 3000+ real desktop and mobile devices online.
Try LambdaTest Now !!
Get 100 minutes of automation test minutes FREE!!
Was this article helpful?
