How to use auto_upload_to_openbenchmarking method of pts_test_run_manager class

Best Phoronix-test-suite code snippet using pts_test_run_manager.auto_upload_to_openbenchmarking

pts_test_run_manager.php

Source:pts_test_run_manager.php Github

copy

Full Screen

...34 protected $prompt_save_results = true;35 protected $post_run_message = null;36 protected $pre_run_message = null;37 protected $allow_sharing_of_results = true;38 protected $auto_upload_to_openbenchmarking = false;39 protected $openbenchmarking_results_data = false;40 protected $do_dynamic_run_count = false;41 protected $dynamic_run_count_on_length_or_less;42 protected $dynamic_run_count_std_deviation_threshold;43 protected $dynamic_run_count_export_script;44 protected $multi_test_stress_run = false;45 protected $allow_test_cache_share = true;46 protected $skip_post_execution_options = false;47 protected $drop_noisy_results = false;48 protected $test_subset = false;49 protected static $test_run_process_active = false;50 protected $batch_mode = false;51 protected $auto_mode = false;52 public $DEBUG_no_test_execution_just_result_parse = false;53 public $benchmark_log = null;54 public $test_run_success_counter = 0;55 public $remove_tests_on_completion = false;56 public function __construct($batch_mode = false, $auto_mode = false)57 {58 $this->do_dynamic_run_count = pts_config::read_bool_config('PhoronixTestSuite/Options/TestResultValidation/DynamicRunCount', 'TRUE') && pts_env::read('FORCE_TIMES_TO_RUN') == false;59 $this->dynamic_run_count_on_length_or_less = 60; //pts_config::read_user_config('PhoronixTestSuite/Options/TestResultValidation/LimitIncreasingRunCountForTestsOverLength', 60);60 $this->dynamic_run_count_std_deviation_threshold = pts_config::read_user_config('PhoronixTestSuite/Options/TestResultValidation/StandardDeviationThreshold', 3.0);61 $this->dynamic_run_count_export_script = pts_config::read_user_config('PhoronixTestSuite/Options/TestResultValidation/ExportResultsTo', null);62 $this->drop_noisy_results = pts_config::read_bool_config('PhoronixTestSuite/Options/TestResultValidation/DropNoisyResults', 'FALSE');63 $this->result_file = new pts_result_file(null);64 if($batch_mode)65 {66 $this->set_batch_mode($batch_mode);67 }68 // 1/true is normal auto mode, 2 = auto + default benchmark mode69 $this->auto_mode = $auto_mode;70 $this->benchmark_log = new pts_logger(null, 'phoronix-test-suite-benchmark.log');71 $this->test_run_success_counter = 0;72 $this->remove_tests_on_completion = pts_config::read_bool_config('PhoronixTestSuite/Options/Testing/RemoveTestInstallOnCompletion', 'FALSE') || pts_env::read('REMOVE_TESTS_ON_COMPLETION');73 pts_module_manager::module_process('__run_manager_setup', $this);74 }75 public function is_interactive_mode()76 {77 return $this->auto_mode == false && $this->batch_mode == false;78 }79 public function do_dynamic_run_count()80 {81 return $this->do_dynamic_run_count;82 }83 public function allow_test_cache_share()84 {85 return $this->allow_test_cache_share;86 }87 public function disable_dynamic_run_count()88 {89 $this->do_dynamic_run_count = false;90 }91 public function auto_upload_to_openbenchmarking($do = true)92 {93 $this->auto_upload_to_openbenchmarking = ($do == true);94 }95 public function do_skip_post_execution_options()96 {97 $this->skip_post_execution_options = true;98 }99 public function increase_run_count_check(&$test_run_request, &$active_result_buffer, $scheduled_times_to_run, $time_test_started = 0)100 {101 // returning false here will not yield extra test run, returning true will yield additional test run, returning -1 will abort/not-save current test result102 if(max($active_result_buffer->results) > 100)103 {104 // On some tests due to handling bugs, "1" will get print as a result when clearly shouldn't be... This check seeks to clear those out and then increase the run count105 // TODO XXX this could potentially be expanded in the future where if given result is 1000x difference from other results, just outright clear?106 $was_hit = false;107 foreach($active_result_buffer->results as $i => $r)108 {109 if($r === 1)110 {111 // Clear out the result112 unset($active_result_buffer->results[$i]);113 }114 }115 if($was_hit)116 {117 return true;118 }119 }120 if($time_test_started && ($min_duration = pts_env::read('FORCE_MIN_DURATION_PER_TEST')) != false && is_numeric($min_duration) && is_numeric($min_duration) > 0)121 {122 // FORCE_MIN_DURATION_PER_TEST if wanting to force a test to run at least for a given amount of time (minutes)123 $time_test_elapsed_so_far = microtime(true) - $time_test_started;124 if($time_test_elapsed_so_far < ($min_duration * 60))125 {126 return true;127 }128 }129 // Compute average time taking per test run (in seconds)130 $avg_test_run_time = pts_math::arithmetic_mean($test_run_request->test_run_times);131 // First make sure this test doesn't take too long to run where we don't want dynamic handling132 if(floor($avg_test_run_time / 60) > $this->dynamic_run_count_on_length_or_less)133 {134 // The default value to not deal with dynamic run counts is when greater than 1 hour (60 minutes)135 // For tests taking an enormous amount of time, by default don't increase run count...136 return false;137 }138 // Determine if results are statistically significant, otherwise up the run count139 $std_dev = pts_math::percent_standard_deviation($active_result_buffer->results);140 if($std_dev >= $this->dynamic_run_count_std_deviation_threshold || pts_math::values_outside_three_sigma_limits($active_result_buffer->results))141 {142 static $test_run_pos; // keeping track of run index for what test in the run queue we are at143 static $run_std_devs; // an array of standard deviations up to this point for the current test144 $times_already_ran = count($active_result_buffer->results); // times test has ran so far145 if($this->test_run_pos != $test_run_pos)146 {147 // We're now onto a new test so clear out the array148 $test_run_pos = $this->test_run_pos;149 $run_std_devs = array();150 }151 $run_std_devs[$times_already_ran] = $std_dev;152 if($avg_test_run_time < 120)153 {154 // If test run time is 2 minutes or less, safely use a 4x multiple for how many times to run for statistical accuracy...155 $maximum_times_to_run = $scheduled_times_to_run * 4;156 }157 else if($avg_test_run_time < 240)158 {159 // If test run time is 4 minutes or less, safely use a 3x multiple for how many times to run for statistical accuracy...160 $maximum_times_to_run = $scheduled_times_to_run * 3;161 }162 else163 {164 // For longer running tests, just consider going up to 2x original expected run count165 $maximum_times_to_run = $scheduled_times_to_run * 2;166 }167 // If we haven't reached scheduled times to run x 2, increase count straight away168 if($times_already_ran < $maximum_times_to_run)169 {170 return true;171 }172 else if($times_already_ran < ($maximum_times_to_run + $scheduled_times_to_run))173 {174 // More aggressive determination whether to still keep increasing the run count beyond the expected maximum...175 $first_and_last_diff_in_deviation = abs(pts_arrays::first_element($run_std_devs) - pts_arrays::last_element($run_std_devs));176 // Increasing the run count at least if it looks to be helping...177 if($first_and_last_diff_in_deviation < (pts_arrays::first_element($run_std_devs) / 2))178 {179 // If we are at least making progress in the right direction, increase the run count some more180 return true;181 }182 // could add more checks and take better advantage of the array of data to better determine if it's still worth increasing183 }184 }185 // Check to see if there is an external/custom script to export the results to in determining whether results are valid186 if(($ex_file = $this->dynamic_run_count_export_script) != null && is_executable($ex_file) || is_executable(($ex_file = PTS_USER_PATH . $this->dynamic_run_count_export_script)))187 {188 $exit_status = trim(shell_exec($ex_file . ' ' . $active_result_buffer->get_values_as_string() . ' > /dev/null 2>&1; echo $?'));189 switch($exit_status)190 {191 case 1:192 // Run the test again193 return true;194 case 2:195 // Results are bad, abandon testing and do not record results196 return -1;197 case 0:198 default:199 // Return was 0 or something else, results are valid, or was some other exit status200 break;201 }202 }203 // see if we should be dropping noisy results204 if($this->drop_noisy_results && $std_dev > 25.0)205 {206 // drop the result from being saved as the noise level exceeds threshold207 return -1;208 }209 // No reason to increase the run count with none of the previous checks requesting otherwise210 return false;211 }212 protected function add_test_result_object(&$test_result)213 {214 $hash = $test_result->get_comparison_hash(true, false);215 if(!isset($this->hashes_of_tests_to_run[$hash]))216 {217 if($test_result->test_profile->test_installation == false || $test_result->test_profile->test_installation->is_installed() == false)218 {219 // Test is not installed, see if should upgrade to new minor release220 $tp = pts_openbenchmarking_client::test_profile_newer_minor_version_available($test_result->test_profile);221 $tests_missing = array();222 if($tp && $this->cleanup_test_profile_valid($tp, $tests_missing, false))223 {224 pts_client::$display->display_interrupt_message('Using ' . $tp->get_identifier() . ' in place of ' . $test_result->test_profile->get_identifier());225 $current_overrides = $test_result->test_profile->get_override_values();226 $test_result->test_profile = $tp;227 $test_result->test_profile->set_override_values($current_overrides);228 }229 }230 if($this->validate_test_to_run($test_result->test_profile))231 {232 $this->hashes_of_tests_to_run[$hash] = $hash;233 $this->tests_to_run[] = $test_result;234 }235 }236 }237 public function get_tests_to_run()238 {239 return $this->tests_to_run;240 }241 public function get_tests_to_run_identifiers()242 {243 $identifiers = array();244 foreach($this->tests_to_run as &$test_run_request)245 {246 $identifiers[] = $test_run_request->test_profile->get_identifier();247 }248 array_unique($identifiers);249 return $identifiers;250 }251 public function get_estimated_run_time($index = -1)252 {253 if($index == -1)254 {255 $index = $this->last_test_run_index;256 }257 $already_added = array();258 $estimated_time = 0;259 foreach(array_slice($this->tests_to_run, $index) as $test_run_request)260 {261 if($test_run_request->test_profile->has_test_options() == false && in_array($test_run_request->test_profile->get_identifier(), $already_added))262 {263 continue;264 }265 $estimated_time += $test_run_request->get_estimated_run_time();266 $already_added[] = $test_run_request->test_profile->get_identifier();267 }268 return $estimated_time;269 }270 public function get_percent_complete()271 {272 return round($this->last_test_run_index / count($this->tests_to_run) * 100);273 }274 public function get_test_to_run($index)275 {276 $this->last_test_run_index = $index;277 return is_numeric($index) && isset($this->tests_to_run[$index]) ? $this->tests_to_run[$index] : false;278 }279 public function get_test_count()280 {281 return count($this->tests_to_run);282 }283 public function force_results_save()284 {285 $this->force_save_results = true;286 }287 public function do_save_results()288 {289 return $this->file_name != null;290 }291 public function get_file_name()292 {293 return $this->file_name;294 }295 public function get_title()296 {297 return $this->file_name_title;298 }299 public function get_results_identifier()300 {301 return $this->results_identifier;302 }303 public function get_results_identifier_simplified()304 {305 return pts_strings::simplify_string_for_file_handling($this->get_results_identifier());306 }307 public function get_description()308 {309 return $this->run_description;310 }311 public function get_notes()312 {313 return null; // TODO: Not Yet Implemented314 }315 public function get_internal_tags()316 {317 return null;318 }319 public function get_reference_id()320 {321 return null;322 }323 public function get_preset_environment_variables()324 {325 return pts_module_manager::var_store_string();326 }327 public function result_already_contains_identifier()328 {329 $contains = false;330 foreach($this->result_file->get_systems() as $s)331 {332 if($s->get_identifier() == $this->results_identifier)333 {334 $contains = true;335 break;336 }337 }338 return $contains;339 }340 public function set_save_name($save_name, $is_new_save = true)341 {342 if(empty($save_name))343 {344 $save_name = date('Y-m-d-Hi', pts_client::current_time());345 }346 $this->file_name = self::clean_save_name($save_name, $is_new_save);347 $this->file_name_title = $save_name;348 $this->force_save_results = true;349 $this->result_file = new pts_result_file($this->file_name);350 $this->benchmark_log->log('SAVE IDENTIFIER: ' . $this->file_name);351 $this->is_new_result_file = $this->result_file->get_system_count() == 0;352 return $this->file_name;353 }354 public function set_results_identifier($identifier)355 {356 $this->results_identifier = self::clean_results_identifier($identifier);357 }358 public function prompt_save_name()359 {360 if($this->file_name != null)361 {362 return $this->file_name;363 }364 // Prompt to save a file when running a test365 $save_name = null;366 if(($env = pts_env::read('TEST_RESULTS_NAME')))367 {368 $save_name = $env;369 //echo 'Saving Results To: ' . $proposed_name . PHP_EOL;370 }371 if(!$this->batch_mode || $this->batch_mode['PromptSaveName'])372 {373 $is_reserved_word = false;374 // Be of help to the user by showing recently saved test results375 if($save_name == null)376 {377 pts_tests::recently_saved_results(' ');378 }379 $save_name_length = $save_name != null ? strlen($save_name) : 0;380 while(empty($save_name) || ($is_reserved_word = pts_types::is_test_or_suite($save_name)) || $save_name_length > 126)381 {382 if($is_reserved_word)383 {384 echo PHP_EOL . 'The name of the saved file cannot be the same as a test/suite: ' . $save_name . PHP_EOL;385 $is_reserved_word = false;386 }387 if($save_name_length > 126)388 {389 echo PHP_EOL . 'The name of the saved file must have between 2 and 126 characters in length.' . PHP_EOL;390 }391 $prompt = ' Enter a name for the result file: ';392 if(function_exists('readline') && function_exists('readline_completion_function'))393 {394 pts_user_io::$readline_completion_possibilities = pts_tests::test_results_by_date();395 readline_completion_function(array('pts_user_io', 'readline_completion_handler'));396 $save_name = readline($prompt);397 }398 else399 {400 $save_name = pts_user_io::read_user_input($prompt);401 }402 }403 }404 return $this->set_save_name($save_name);405 }406 public function prompt_results_identifier()407 {408 if(!empty($this->results_identifier))409 {410 return $this->results_identifier;411 }412 // Prompt for a results identifier413 $results_identifier = null;414 $show_identifiers = array();415 $no_repeated_tests = true;416 if(!$this->is_new_result_file)417 {418 // Running on an already-saved result419 $current_identifiers = array();420 $current_hardware = array();421 $current_software = array();422 foreach($this->result_file->get_systems() as $s)423 {424 $current_hardware[] = $s->get_hardware();425 $current_software[] = $s->get_software();426 $current_identifiers[] = $s->get_identifier();427 }428 $hashes = array();429 foreach($this->result_file->get_result_objects() as $result)430 {431 $hashes[] = $result->get_comparison_hash(true, false);432 }433 foreach($this->tests_to_run as &$run_request)434 {435 if($run_request instanceof pts_test_result && in_array($run_request->get_comparison_hash(true, false), $hashes))436 {437 $no_repeated_tests = false;438 break;439 }440 }441 }442 else443 {444 // Fresh run445 $current_identifiers = array();446 $current_hardware = array();447 $current_software = array();448 }449 if((!$this->batch_mode || $this->batch_mode['PromptForTestIdentifier']) && !$this->auto_mode)450 {451 if(count($current_identifiers) > 0)452 {453 echo PHP_EOL . pts_client::cli_just_bold('Current Test Identifiers:') . PHP_EOL;454 echo pts_user_io::display_text_list($current_identifiers);455 echo PHP_EOL;456 }457 $times_tried = 0;458 do459 {460 if($times_tried == 0 && ($env_identifier = pts_env::read('TEST_RESULTS_IDENTIFIER')))461 {462 $results_identifier = isset($env_identifier) ? self::clean_results_identifier($env_identifier) : null;463 echo 'Test Identifier: ' . $results_identifier . PHP_EOL;464 }465 else466 {467 $prompt = ' Enter a unique name to describe this test run / configuration: ';468 if(function_exists('readline') && function_exists('readline_completion_function'))469 {470 pts_user_io::$readline_completion_possibilities = array_map(array('pts_strings', 'trim_search_query'), array_merge(phodevi::system_hardware(false), phodevi::system_software(false)));471 readline_completion_function(array('pts_user_io', 'readline_completion_handler'));472 $results_identifier = readline($prompt);473 }474 else475 {476 $results_identifier = pts_user_io::read_user_input($prompt);477 }478 $results_identifier = self::clean_results_identifier($results_identifier);479 }480 $times_tried++;481 $identifier_pos = (($p = array_search($results_identifier, $current_identifiers)) !== false ? $p : -1);482 }483 while((!$no_repeated_tests && $identifier_pos != -1) || (isset($current_hardware[$identifier_pos]) && $current_hardware[$identifier_pos] != phodevi::system_hardware(true)) || (isset($current_software[$identifier_pos]) && $current_software[$identifier_pos] != phodevi::system_software(true)));484 }485 else if(($env_identifier = pts_env::read('TEST_RESULTS_IDENTIFIER')))486 {487 $results_identifier = self::clean_results_identifier($env_identifier);488 }489 if(empty($results_identifier))490 {491 $results_identifier = $this->auto_generate_results_identifier();492 }493 $this->results_identifier = $results_identifier;494 return $this->results_identifier;495 }496 public function auto_generate_results_identifier()497 {498 // If the save result identifier is empty, try to come up with something based upon the tests being run.499 $results_identifier = null;500 $subsystem_r = array();501 $subsystems_to_test = $this->subsystems_under_test();502 if(!$this->is_new_result_file)503 {504 $result_file_intent = pts_result_file_analyzer::analyze_result_file_intent($this->result_file);505 if(is_array($result_file_intent) && $result_file_intent[0] != 'Unknown')506 {507 array_unshift($subsystems_to_test, $result_file_intent[0]);508 }509 }510 foreach($subsystems_to_test as $subsystem)511 {512 $components = pts_result_file_analyzer::system_component_string_to_array(phodevi::system_hardware(true) . ', ' . phodevi::system_software(true));513 if($subsystem != null && isset($components[$subsystem]))514 {515 $subsystem_name = trim(pts_strings::trim_search_query($components[$subsystem]));516 if(!empty($subsystem_name) && phodevi::is_vendor_string($subsystem_name) && !in_array($subsystem_name, $subsystem_r))517 {518 $subsystem_r[] = $subsystem_name;519 }520 if(isset($subsystem_r[2]) || isset($subsystem_name[19]))521 {522 break;523 }524 }525 }526 if(isset($subsystem_r[0]))527 {528 $results_identifier = implode(' - ', $subsystem_r);529 }530 if(empty($results_identifier) && !$this->batch_mode)531 {532 $results_identifier = phodevi::read_property('cpu', 'model') . ' - ' . phodevi::read_property('gpu', 'model') . ' - ' . phodevi::read_property('motherboard', 'identifier');533 }534 if(strlen($results_identifier) > 55)535 {536 $results_identifier = substr($results_identifier, 0, 54);537 $results_identifier = substr($results_identifier, 0, strrpos($results_identifier, ' '));538 }539 if(empty($results_identifier))540 {541 $results_identifier = date('Y-m-d H:i', pts_client::current_time());542 }543 $this->results_identifier = $results_identifier;544 $this->benchmark_log->log('RESULTS IDENTIFIER: ' . $results_identifier);545 return $results_identifier;546 }547 public static function clean_results_identifier($results_identifier)548 {549 $results_identifier = trim(pts_client::swap_variables($results_identifier, array('pts_test_run_manager', 'user_run_save_variables')));550 $results_identifier = pts_strings::remove_redundant(pts_strings::keep_in_string($results_identifier, pts_strings::CHAR_LETTER | pts_strings::CHAR_NUMERIC | pts_strings::CHAR_DASH | pts_strings::CHAR_UNDERSCORE | pts_strings::CHAR_COLON | pts_strings::CHAR_COMMA | pts_strings::CHAR_SLASH | pts_strings::CHAR_SPACE | pts_strings::CHAR_DECIMAL | pts_strings::CHAR_AT | pts_strings::CHAR_PLUS | pts_strings::CHAR_SEMICOLON | pts_strings::CHAR_EQUAL), ' ');551 return $results_identifier;552 }553 public function get_test_run_position()554 {555 return ($this->get_test_count() * ($this->loop_run_pos - 1)) + $this->test_run_pos + 1;556 }557 public function get_test_run_count_reported()558 {559 return $this->test_run_count;560 }561 public function call_test_runs()562 {563 // Create a lock564 $lock_path = pts_client::temporary_directory() . '/phoronix-test-suite.active';565 pts_client::create_lock($lock_path);566 if($this->pre_run_message != null)567 {568 pts_client::$display->display_interrupt_message($this->pre_run_message);569 }570 // Hook into the module framework571 self::$test_run_process_active = true;572 pts_module_manager::module_process('__pre_run_process', $this);573 pts_file_io::unlink(PTS_USER_PATH . 'halt-testing');574 pts_file_io::unlink(PTS_USER_PATH . 'skip-test');575 $continue_test_flag = true;576 $tests_to_run_count = $this->get_test_count();577 pts_client::$display->test_run_process_start($this);578 $total_loop_count = (($t = pts_env::read('TOTAL_LOOP_COUNT')) && is_numeric($t) && $t > 0) ? $t : 1;579 $total_loop_time = (($t = pts_env::read('TOTAL_LOOP_TIME')) && is_numeric($t) && $t > 9) ? ($t * 60) : -1;580 $loop_end_time = $total_loop_time != -1 ? (time() + $total_loop_time) : false;581 $this->test_run_count = ($tests_to_run_count * $total_loop_count);582 for($loop = 1; $loop <= $total_loop_count && $continue_test_flag; $loop++)583 {584 $this->loop_run_pos = $loop;585 for($i = 0; $i < $tests_to_run_count && $continue_test_flag; $i++)586 {587 $this->test_run_pos = $i;588 $continue_test_flag = $this->process_test_run_request($i);589 if($continue_test_flag === 'SKIP')590 {591 $continue_test_flag = true;592 continue;593 }594 if($this->remove_tests_on_completion)595 {596 // Remove the installed test if it's no longer needed in this run queue597 $this_test_profile_identifier = $this->get_test_to_run($this->test_run_pos)->test_profile->get_identifier();598 $still_in_queue = false;599 for($j = ($this->test_run_pos + 1); $j < $tests_to_run_count && $still_in_queue == false; $j++)600 {601 if($this->get_test_to_run($j)->test_profile->get_identifier() == $this_test_profile_identifier)602 {603 $still_in_queue = true;604 }605 }606 if($still_in_queue == false)607 {608 pts_tests::remove_installed_test($this->get_test_to_run($this->test_run_pos)->test_profile);609 }610 }611 if($loop_end_time)612 {613 if(time() > $loop_end_time)614 {615 $continue_test_flag = false;616 }617 else if($this->test_run_count == ($i + 1))618 {619 // There's still time remaining so increase the run count....620 $this->test_run_count += $tests_to_run_count;621 }622 }623 }624 }625 pts_file_io::unlink(PTS_SAVE_RESULTS_PATH . $this->get_file_name() . '/active.xml');626 foreach($this->tests_to_run as &$run_request)627 {628 // Remove cache shares629 foreach(pts_file_io::glob($run_request->test_profile->get_install_dir() . 'cache-share-*.pt2so') as $cache_share_file)630 {631 unlink($cache_share_file);632 }633 }634 pts_triggered_system_events::test_requested_queued_reboot_check();635 if($this->post_run_message != null)636 {637 pts_client::$display->display_interrupt_message($this->post_run_message);638 }639 self::$test_run_process_active = -1;640 pts_module_manager::module_process('__post_run_process', $this);641 pts_client::release_lock($lock_path);642 // Report any tests that failed to properly run643 if(pts_client::is_debug_mode() || $this->get_test_count() > 3)644 {645 if(count($this->failed_tests_to_run) > 0)646 {647 echo PHP_EOL . PHP_EOL . 'The following tests failed to properly run:' . PHP_EOL . PHP_EOL;648 foreach($this->failed_tests_to_run as &$run_request)649 {650 echo ' - ' . $run_request->test_profile->get_identifier() . ($run_request->get_arguments_description() != null ? ': ' . $run_request->get_arguments_description() : null) . PHP_EOL;651 }652 echo PHP_EOL;653 }654 }655 }656 public static function test_run_process_active()657 {658 return self::$test_run_process_active = true;659 }660 public function process_test_run_request($run_index)661 {662 $result = false;663 if($this->do_save_results())664 {665 $this->result_file->get_xml(PTS_SAVE_RESULTS_PATH . $this->get_file_name() . '/composite.xml');666 }667 if(is_object($run_index))668 {669 $test_run_request = $run_index;670 $run_index = 0;671 }672 else673 {674 $test_run_request = $this->get_test_to_run($run_index);675 }676 if($test_run_request == false)677 {678 return;679 }680 if($this->result_file->has_matching_test_and_run_identifier($test_run_request, $this->get_results_identifier()) && pts_env::read('TOTAL_LOOP_COUNT') == false && pts_env::read('TOTAL_LOOP_TIME') == false)681 {682 // There already is a match for this test in this particular result buffer683 // except if using one of the loop controls where it may be repeated...684 return true;685 }686 $skip_tests_with_args = ($e = pts_env::read('SKIP_TESTS_HAVING_ARGS')) ? pts_strings::comma_explode($e) : false;687 if($skip_tests_with_args)688 {689 foreach($skip_tests_with_args as $skip_test_if_arg_matches)690 {691 if(stripos($test_run_request->get_arguments_description(), $skip_test_if_arg_matches) !== false)692 {693 return true;694 }695 }696 }697 if(($run_index != 0 && count(pts_file_io::glob($test_run_request->test_profile->get_install_dir() . 'cache-share-*.pt2so')) == 0))698 {699 // Sleep for six seconds between tests by default700 sleep(6);701 }702 $this->benchmark_log->log('Executing Test: ' . $test_run_request->test_profile->get_identifier());703 $test_successful = pts_test_execution::run_test($this, $test_run_request);704 if(pts_file_io::unlink(PTS_USER_PATH . 'halt-testing'))705 {706 // Stop the testing process entirely707 return false;708 }709 else if(pts_file_io::unlink(PTS_USER_PATH . 'skip-test'))710 {711 // Just skip the current test and do not save the results, but continue testing712 return 'SKIP';713 }714 else if(pts_env::read('LIMIT_ELAPSED_TEST_TIME') > 0 && (PTS_INIT_TIME + (pts_env::read('LIMIT_ELAPSED_TEST_TIME') * 60)) > time())715 {716 // Allocated amount of time has expired717 return false;718 }719 if($test_successful == false && $test_run_request->test_profile->get_identifier() != null)720 {721 $this->failed_tests_to_run[] = $test_run_request;722 }723 pts_module_manager::module_process('__post_test_run_process', $this->result_file);724 return true;725 }726 public static function process_json_report_attributes(&$test_run_request, $report_error = null)727 {728 // XXX : add to attributes JSON here729 $json_report_attributes = null;730 if(is_object($test_run_request->test_profile->test_installation))731 {732 if(($t = $test_run_request->test_profile->test_installation->get_compiler_data()))733 {734 $json_report_attributes['compiler-options'] = $t;735 }736 if(($t = $test_run_request->test_profile->test_installation->get_install_footnote()))737 {738 $json_report_attributes['install-footnote'] = $t;739 }740 }741 if(($t = $test_run_request->active->get_min_result()) != 0)742 {743 $json_report_attributes['min-result'] = $t;744 }745 if(($t = $test_run_request->active->get_max_result()) != 0)746 {747 $json_report_attributes['max-result'] = $t;748 }749 if(!empty($test_run_request->test_run_times))750 {751 $json_report_attributes['test-run-times'] = implode(':', $test_run_request->test_run_times);752 }753 if(!empty($report_error))754 {755 $json_report_attributes['error'] = $report_error;756 }757 return $json_report_attributes;758 }759 public static function clean_save_name($input, $is_new_save = true)760 {761 $input = pts_client::swap_variables($input, array('pts_test_run_manager', 'user_run_save_variables'));762 $input = pts_strings::remove_redundant(pts_strings::keep_in_string(str_replace(' ', '-', trim($input)), pts_strings::CHAR_LETTER | pts_strings::CHAR_NUMERIC | pts_strings::CHAR_DASH), '-');763 if($is_new_save)764 {765 $input = strtolower($input);766 }767 if(strlen($input) > 126)768 {769 $input = substr($input, 0, 126);770 }771 return $input;772 }773 public function initial_checks(&$to_run, $override_display_mode = false)774 {775 // Refresh the pts_client::$display in case we need to run in debug mode776 if(pts_client::$display == false || !(pts_client::$display instanceof pts_websocket_display_mode))777 {778 pts_client::init_display_mode($override_display_mode);779 }780 $to_run = pts_types::identifiers_to_objects($to_run);781 if($this->batch_mode && $this->batch_mode['Configured'] == false && !$this->auto_mode)782 {783 trigger_error('The batch mode must first be configured.' . PHP_EOL . 'To configure, run phoronix-test-suite batch-setup', E_USER_ERROR);784 return false;785 }786 if(!is_writable(pts_client::test_install_root_path()))787 {788 trigger_error('The test installation directory is not writable.' . PHP_EOL . 'Location: ' . pts_client::test_install_root_path(), E_USER_ERROR);789 return false;790 }791 $mount_options = phodevi::read_property('disk', 'mount-options');792 if(isset($mount_options['mount-options']) && strpos($mount_options['mount-options'], 'noexec') !== false)793 {794 trigger_error('The test installation directory is on a file-system mounted with the \'noexec\' mount option. Re-mount the file-system appropriately or change the Phoronix Test Suite user configuration file to point to an alternative mount point.' . PHP_EOL . 'Location: ' . pts_client::test_install_root_path(), E_USER_ERROR);795 return false;796 }797 // Cleanup tests to run798 if($this->cleanup_tests_to_run($to_run) == false)799 {800 return false;801 }802 else if(count($to_run) == 0)803 {804 //trigger_error('You must enter at least one test, suite, or result identifier to run.', E_USER_ERROR);805 return false;806 }807 return true;808 }809 public function pre_execution_process()810 {811 if($this->is_new_result_file || $this->result_already_contains_identifier() == false)812 {813 $this->result_file->set_title($this->file_name_title);814 $this->result_file->set_description($this->run_description);815 $this->result_file->set_notes($this->get_notes());816 $this->result_file->set_internal_tags($this->get_internal_tags());817 $this->result_file->set_reference_id($this->get_reference_id());818 $this->result_file->set_preset_environment_variables($this->get_preset_environment_variables());819 // TODO XXX JSON In null and notes820 $json_attr = $this->generate_json_system_attributes();821 $sys = new pts_result_file_system($this->results_identifier, phodevi::system_hardware(true), phodevi::system_software(true), $json_attr, pts_client::current_user(), null, date('Y-m-d H:i:s', pts_client::current_time()), PTS_VERSION, $this->result_file);822 $this->result_file->add_system($sys);823 }824 if($this->do_save_results())825 {826 pts_client::setup_test_result_directory($this->get_file_name());827 }828 }829 protected function generate_json_system_attributes()830 {831 $test_external_dependencies = array();832 $test_hardware_types = array();833 $test_internal_tags = array();834 foreach($this->tests_to_run as &$test_to_run)835 {836 $test_external_dependencies = array_merge($test_external_dependencies, $test_to_run->test_profile->get_external_dependencies());837 $test_internal_tags = array_merge($test_internal_tags, $test_to_run->test_profile->get_internal_tags());838 pts_arrays::unique_push($test_hardware_types, $test_to_run->test_profile->get_test_hardware_type());839 }840 return self::pull_test_notes(false, $test_external_dependencies, $test_internal_tags, $test_hardware_types);841 }842 public static function pull_test_notes($show_all = false, $test_external_dependencies = array(), $test_internal_tags = array(), $test_hardware_types = array())843 {844 $notes = null;845 if($show_all || in_array('build-utilities', $test_external_dependencies))846 {847 // So compiler tests were run....848 $test = false;849 $compiler_mask_dir = pts_test_installer::create_compiler_mask($test);850 if($compiler_mask_dir && is_executable($compiler_mask_dir . 'cc'))851 {852 $compiler_configuration = phodevi_system::sw_compiler_build_configuration($compiler_mask_dir . 'cc');853 pts_file_io::delete($compiler_mask_dir, null, true);854 if(!empty($compiler_configuration))855 {856 $notes['compiler-configuration'] = $compiler_configuration;857 }858 }859 }860 if($show_all || in_array('OpenCL', $test_internal_tags) || in_array('opencl', $test_external_dependencies))861 {862 // So OpenCL tests were run....863 $gpu_compute_cores = phodevi::read_property('gpu', 'compute-cores');864 if($gpu_compute_cores > 0)865 {866 $notes['graphics-compute-cores'] = $gpu_compute_cores;867 }868 }869 if($show_all || in_array('Disk', $test_hardware_types))870 {871 // A disk test was run so report some disk information...872 $disk_scheduler = phodevi::read_property('disk', 'scheduler');873 if($disk_scheduler)874 {875 $notes['disk-scheduler'] = $disk_scheduler;876 }877 $mount_options = phodevi::read_property('disk', 'mount-options');878 if(isset($mount_options['mount-options']) && $mount_options['mount-options'] != null)879 {880 $notes['disk-mount-options'] = $mount_options['mount-options'];881 }882 $extra = phodevi::read_property('disk', 'extra-disk-details');883 if($extra != null)884 {885 $notes['disk-details'] = $extra;886 }887 }888 if(true || $show_all || in_array('Processor', $test_hardware_types) || in_array('System', $test_hardware_types))889 {890 // makes sense always reporting the CPU scaling governor891 $scaling_governor = phodevi::read_property('cpu', 'scaling-governor');892 if($scaling_governor)893 {894 $notes['cpu-scaling-governor'] = $scaling_governor;895 }896 $cpu_microcode = phodevi::read_property('cpu', 'microcode-version');897 if($cpu_microcode)898 {899 $notes['cpu-microcode'] = $cpu_microcode;900 }901 if(phodevi::is_linux() && pts_client::executable_in_path('thermald') && (pts_client::is_process_running('thermald') || phodevi_linux_parser::systemctl_active('thermald')))902 {903 $thermald_version = trim(shell_exec('thermald --version 2>/dev/null'));904 if(!empty($thermald_version) && pts_strings::is_version($thermald_version))905 {906 $notes['cpu-thermald'] = $thermald_version;907 }908 }909 // POWER processors have configurable SMT, 1-8 per core.910 $smt = phodevi::read_property('cpu', 'smt');911 if($smt)912 {913 $notes['cpu-smt'] = $smt;914 }915 $cpu_pm = phodevi::read_property('cpu', 'power-management');916 if($cpu_pm)917 {918 $notes['cpu-pm'] = $cpu_pm;919 }920 $platform_profile = phodevi::read_property('system', 'platform-profile');921 if($platform_profile)922 {923 $notes['platform-profile'] = $platform_profile;924 }925 }926 if($show_all || in_array('Graphics', $test_hardware_types))927 {928 $accel_2d = phodevi::read_property('gpu', '2d-acceleration');929 if($accel_2d)930 {931 $notes['graphics-2d-acceleration'] = $accel_2d;932 }933 $aa = phodevi::read_property('gpu', 'aa-level');934 if($aa)935 {936 $notes['graphics-aa'] = $aa;937 }938 $af = phodevi::read_property('gpu', 'af-level');939 if($af)940 {941 $notes['graphics-af'] = $af;942 }943 $oc_offset = phodevi::read_property('gpu', 'oc-offset-string');944 if(!empty($oc_offset))945 {946 $notes['graphics-oc'] = $oc_offset;947 }948 $bar1_visible_vram_rebar = phodevi::read_property('gpu', 'bar1-visible-vram');949 if(!empty($bar1_visible_vram_rebar))950 {951 $notes['bar1-visible-vram'] = $bar1_visible_vram_rebar;952 }953 }954 if($show_all || phodevi::read_property('system', 'kernel-parameters'))955 {956 $notes['kernel-parameters'] = phodevi::read_property('system', 'kernel-parameters');957 }958 if($show_all || phodevi::read_property('system', 'kernel-extra-details'))959 {960 $notes['kernel-extra-details'] = phodevi::read_property('system', 'kernel-extra-details');961 }962 if($show_all || phodevi::read_property('system', 'environment-variables', false))963 {964 $notes['environment-variables'] = phodevi::read_property('system', 'environment-variables', false);965 }966 if($show_all || in_array('Java', $test_internal_tags) || in_array('java', $test_external_dependencies))967 {968 $notes['java'] = phodevi::read_property('system', 'java-version');969 }970 if($show_all || in_array('Python', $test_internal_tags) || in_array('python', $test_external_dependencies))971 {972 $notes['python'] = phodevi::read_property('system', 'python-version');973 }974 if(in_array('wine', $test_external_dependencies))975 {976 phodevi_system::$report_wine_override = true;977 }978 $notes['security'] = phodevi::read_property('system', 'security-features');979 foreach($notes as $key => $value)980 {981 if(empty($value))982 {983 unset($notes[$key]);984 }985 }986 return $notes;987 }988 public function post_execution_process()989 {990 $this->benchmark_log->log('Test Run Process Ended');991 if($this->do_save_results() && !$this->skip_post_execution_options)992 {993 // Save the results994 echo PHP_EOL;995 pts_module_manager::module_process('__event_results_process', $this);996 pts_client::save_test_result($this->get_file_name() . '/composite.xml', $this->result_file->get_xml(), true, $this->results_identifier);997 pts_module_manager::module_process('__event_results_saved', $this);998 if($this->test_run_success_counter == 0 && $this->is_new_result_file)999 {1000 return false;1001 }1002 }1003 if($this->test_run_success_counter > 3 && pts_config::read_bool_config('PhoronixTestSuite/Options/Testing/ShowPostRunStatistics', 'TRUE'))1004 {1005 // Show any post run statistics1006 pts_module_manager::module_process('__event_post_run_stats', $this);1007 if($this->result_file->get_system_count() == 2)1008 {1009 $highlights = pts_result_file_analyzer::display_results_baseline_two_way_compare($this->result_file, true, false, true, ' ');1010 if($highlights)1011 {1012 echo ' ' . pts_client::cli_just_bold('Result Highlights') . PHP_EOL;1013 echo $highlights . PHP_EOL;1014 }1015 }1016 if($this->result_file->get_system_count() > 2)1017 {1018 // Display winners and losers1019 echo pts_result_file_analyzer::display_results_wins_losses($this->result_file, $this->get_results_identifier(), ' ') . PHP_EOL;1020 }1021 if($this->result_file->get_system_count() > 1)1022 {1023 echo pts_result_file_analyzer::display_result_file_stats_pythagorean_means($this->result_file, $this->get_results_identifier());1024 }1025 }1026 if($this->do_save_results() && !$this->skip_post_execution_options)1027 {1028 // See if the results should be displayed1029 //echo PHP_EOL . 'Results Saved To: ; . PTS_SAVE_RESULTS_PATH . $this->get_file_name() . ;/composite.xml' . PHP_EOL;1030 if(!$this->auto_mode)1031 {1032 if($this->batch_mode)1033 {1034 if($this->batch_mode['OpenBrowser'])1035 {1036 pts_client::display_result_view($this->result_file, true, null);1037 }1038 }1039 else1040 {1041 if(!phodevi::is_display_server_active() && !defined('PHOROMATIC_PROCESS'))1042 {1043 pts_client::display_result_view($this->result_file, false, 'Do you want to view the text results of the testing');1044 }1045 else1046 {1047 pts_client::display_result_view($this->result_file, false, '');1048 }1049 }1050 }1051 if($this->allow_sharing_of_results && pts_network::internet_support_available() && pts_openbenchmarking::ob_upload_support_available())1052 {1053 if($this->auto_upload_to_openbenchmarking || pts_openbenchmarking_client::auto_upload_results() || pts_config::read_bool_config('PhoronixTestSuite/Options/Testing/AlwaysUploadResultsToOpenBenchmarking', 'FALSE'))1054 {1055 $upload_results = true;1056 }1057 else if($this->batch_mode)1058 {1059 $upload_results = $this->batch_mode['UploadResults'];1060 }1061 else if(!$this->auto_mode)1062 {1063 $upload_results = pts_user_io::prompt_bool_input('Would you like to upload the results to OpenBenchmarking.org', -1);1064 }1065 else1066 {1067 $upload_results = false;...

Full Screen

Full Screen

auto_upload_to_openbenchmarking

Using AI Code Generation

copy

Full Screen

1require_once('pts-core.php');2$test_run_manager = new pts_test_run_manager();3$test_run_manager->auto_upload_to_openbenchmarking();4require_once('pts-core.php');5$test_run_manager = new pts_test_run_manager();6$test_run_manager->auto_upload_to_openbenchmarking();7require_once('pts-core.php');8$test_run_manager = new pts_test_run_manager();9$test_run_manager->auto_upload_to_openbenchmarking();10require_once('pts-core.php');11$test_run_manager = new pts_test_run_manager();12$test_run_manager->auto_upload_to_openbenchmarking();13require_once('pts-core.php');14$test_run_manager = new pts_test_run_manager();15$test_run_manager->auto_upload_to_openbenchmarking();16require_once('pts-core.php');17$test_run_manager = new pts_test_run_manager();18$test_run_manager->auto_upload_to_openbenchmarking();19require_once('pts-core.php');20$test_run_manager = new pts_test_run_manager();21$test_run_manager->auto_upload_to_openbenchmarking();22require_once('pts-core.php');23$test_run_manager = new pts_test_run_manager();24$test_run_manager->auto_upload_to_openbenchmarking();25require_once('pts-core.php');26$test_run_manager = new pts_test_run_manager();27$test_run_manager->auto_upload_to_openbenchmarking();28require_once('

Full Screen

Full Screen

auto_upload_to_openbenchmarking

Using AI Code Generation

copy

Full Screen

1$client = new pts_test_run_manager();2$client->auto_upload_to_openbenchmarking();3$client = new pts_test_run_manager();4$client->auto_upload_to_phoronix_test_suite();5$client = new pts_test_run_manager();6$client->auto_upload_to_phoronix_test_suite();7$client = new pts_test_run_manager();8$client->auto_upload_to_phoronix_test_suite();9$client = new pts_test_run_manager();10$client->auto_upload_to_phoronix_test_suite();11$client = new pts_test_run_manager();12$client->auto_upload_to_phoronix_test_suite();13$client = new pts_test_run_manager();14$client->auto_upload_to_phoronix_test_suite();15$client = new pts_test_run_manager();16$client->auto_upload_to_phoronix_test_suite();17$client = new pts_test_run_manager();18$client->auto_upload_to_phoronix_test_suite();19$client = new pts_test_run_manager();20$client->auto_upload_to_phoronix_test_suite();21$client = new pts_test_run_manager();22$client->auto_upload_to_phoronix_test_suite();

Full Screen

Full Screen

auto_upload_to_openbenchmarking

Using AI Code Generation

copy

Full Screen

1$test_run_manager = new pts_test_run_manager();2$test_run_manager->auto_upload_to_openbenchmarking('2.php');3$test_run_manager = new pts_test_run_manager();4$test_run_manager->auto_upload_to_openbenchmarking('2.php');5$test_run_manager = new pts_test_run_manager();6$test_run_manager->auto_upload_to_openbenchmarking('2.php');7$test_run_manager = new pts_test_run_manager();8$test_run_manager->auto_upload_to_openbenchmarking('2.php');9$test_run_manager = new pts_test_run_manager();10$test_run_manager->auto_upload_to_openbenchmarking('2.php');11$test_run_manager = new pts_test_run_manager();12$test_run_manager->auto_upload_to_openbenchmarking('2.php');13$test_run_manager = new pts_test_run_manager();14$test_run_manager->auto_upload_to_openbenchmarking('2.php');15$test_run_manager = new pts_test_run_manager();16$test_run_manager->auto_upload_to_openbenchmarking('2.php');17$test_run_manager = new pts_test_run_manager();18$test_run_manager->auto_upload_to_openbenchmarking('2.php');

Full Screen

Full Screen

auto_upload_to_openbenchmarking

Using AI Code Generation

copy

Full Screen

1$test_run_manager = new pts_test_run_manager();2$test_run_manager->auto_upload_to_openbenchmarking('2.php', 'My Test Profile', 'My Test Profile Description', 'My Test Profile License', 'My Test Profile Maintainer', 'My Test Profile Maintainer E-Mail');3$test_run_manager = new pts_test_run_manager();4$test_run_manager->auto_upload_to_openbenchmarking('3.php', 'My Test Profile', 'My Test Profile Description', 'My Test Profile License', 'My Test Profile Maintainer', 'My Test Profile Maintainer E-Mail');5$test_run_manager = new pts_test_run_manager();6$test_run_manager->auto_upload_to_openbenchmarking('4.php', 'My Test Profile', 'My Test Profile Description', 'My Test Profile License', 'My Test Profile Maintainer', 'My Test Profile Maintainer E-Mail');7$test_run_manager = new pts_test_run_manager();8$test_run_manager->auto_upload_to_openbenchmarking('5.php', 'My Test Profile', 'My Test Profile Description', 'My Test Profile License', 'My Test Profile Maintainer', 'My Test Profile Maintainer E-Mail');9$test_run_manager = new pts_test_run_manager();10$test_run_manager->auto_upload_to_openbenchmarking('6.php', 'My Test Profile', 'My Test Profile Description', 'My Test Profile License', 'My Test Profile Maintainer', 'My Test Profile Maintainer E-Mail');11$test_run_manager = new pts_test_run_manager();12$test_run_manager->auto_upload_to_openbenchmarking('7.php', 'My Test Profile

Full Screen

Full Screen

auto_upload_to_openbenchmarking

Using AI Code Generation

copy

Full Screen

1$test_run_manager = new pts_test_run_manager();2$results = $test_run_manager->get_test_result_object();3$test_run_manager->auto_upload_to_openbenchmarking($results);4$results = $test_run_manager->get_test_result_object();5print_r($results);6echo $results->test_profile->test_result_buffer->to_table();7 (

Full Screen

Full Screen

Automation Testing Tutorials

Learn to execute automation testing from scratch with LambdaTest Learning Hub. Right from setting up the prerequisites to run your first automation test, to following best practices and diving deeper into advanced test scenarios. LambdaTest Learning Hubs compile a list of step-by-step guides to help you be proficient with different test automation frameworks i.e. Selenium, Cypress, TestNG etc.

LambdaTest Learning Hubs:

YouTube

You could also refer to video tutorials over LambdaTest YouTube channel to get step by step demonstration from industry experts.

Run Phoronix-test-suite automation tests on LambdaTest cloud grid

Perform automation testing on 3000+ real desktop and mobile devices online.

... View sample repo... View DocumentationSign up Free
›
_

Most used method in pts_test_run_manager

Trigger auto_upload_to_openbenchmarking code on LambdaTest Cloud Grid

Execute automation tests with auto_upload_to_openbenchmarking on a cloud-based Grid of 3000+ real browsers and operating systems for both web and mobile applications.

Test now for Free

Try LambdaTest Now !!

Get 100 minutes of automation test minutes FREE!!

...
Sign up with GoogleSign up with Email
Next-Gen App & Browser Testing Cloud

Was this article helpful?

Helpful

NotHelpful