Discover AI-native insights at Testμ Conference‘25

How to use StringTokenScanner class

Best Cucumber Common Library code snippet using StringTokenScanner

GherkinParser.php

Source:GherkinParser.php

...81        $builder = new GherkinDocumentBuilder($source->uri, $this->idGenerator);82        $parser = new Parser($builder);83        return $parser->parse(84            $source->uri,85            new StringTokenScanner($source->data),86            new TokenMatcher(),87        );88    }89    private function createParseError(ParserException $error, string $uri): Envelope90    {91        $ref = new SourceReference(92            uri: $uri,93            location: new MessageLocation(94                line: $error->location->line,95                column: $error->location->column === 0 ? null : $error->location->column,96            ),97        );98        return new Envelope(parseError: new ParseError($ref, $error->getMessage()));99    }...

StringTokenScannerTest.php

Source:StringTokenScannerTest.php

1<?php2declare(strict_types=1);3namespace Cucumber\Gherkin;4use PHPUnit\Framework\TestCase;5final class StringTokenScannerTest extends TestCase6{7    public function testItFindsEofForEmptyStream(): void8    {9        $stream = '';10        $scanner = new StringTokenScanner($stream);11        $token = $scanner->read();12        self::assertTrue($token->isEof());13    }14    public function testItFindsOneTokenPerLineForPopulatedStream(): void15    {16        $contents = "FOO\nBAR\r\nBAZ\n";17        $scanner = new StringTokenScanner($contents);18        $token = $scanner->read();19        self::assertSame('FOO', $token->getTokenValue());20        $token = $scanner->read();21        self::assertSame('BAR', $token->getTokenValue());22        $token = $scanner->read();23        self::assertSame('BAZ', $token->getTokenValue());24        $token = $scanner->read();25        self::assertSame('EOF', $token->getTokenValue());26        self::assertTrue($token->isEof());27    }28    public function testItFindsEmptyLines(): void29    {30        $contents = "FOO\n\nBAR\n";31        $scanner = new StringTokenScanner($contents);32        $token = $scanner->read();33        self::assertSame('FOO', $token->getTokenValue());34        $token = $scanner->read();35        self::assertSame('', $token->getTokenValue());36        $token = $scanner->read();37        self::assertSame('BAR', $token->getTokenValue());38        $token = $scanner->read();39        self::assertSame('EOF', $token->getTokenValue());40        self::assertTrue($token->isEof());41    }42}...

StringTokenScanner.php

Source:StringTokenScanner.php

...8 *9 * If the scanner sees a # language header, it will reconfigure itself dynamically to look for10 * Gherkin keywords for the associated language. The keywords are defined in gherkin-languages.json.11 */12final class StringTokenScanner implements TokenScanner13{14    private const FIRST_LINE_PATTERN = '/^(?<line>.*?)\\r?\\n(?<tail>.*)$/us';15    private int $lineNumber = 0;16    public function __construct(17        private string $source,18    ) {19    }20    public function read(): Token21    {22        if (preg_match(self::FIRST_LINE_PATTERN, $this->source, $matches)) {23            $line = $matches['line'];24            $this->source = $matches['tail'];25        } else { // it did not contain a line break26            $line = $this->source;...

gherkin-generate-tokens

Source: gherkin-generate-tokens

...10array_shift($argv);11foreach($argv as $fileName) {12    $result = $parser->parse(13            $fileName,14            new StringTokenScanner(file_get_contents($fileName)),15            new TokenMatcher(),16    );17    echo $result;18}...

StringTokenScanner

Using AI Code Generation

1include 'vendor/autoload.php';2use Cucumber\Gherkin\StringTokenScanner;3$scanner = new StringTokenScanner("I have 2 apples");4$scanner->readUntil("have");5$scanner->read("have");6$scanner->readUntil("apples");7$scanner->read("apples");8echo $scanner->getReadCount();9echo $scanner->getReadPosition();10echo $scanner->getReadString();11echo $scanner->getReadToken();12echo $scanner->getReadTokens();13echo $scanner->getRemainingString();14echo $scanner->getRemainingTokens();15echo $scanner->getRest();16echo $scanner->getRestUntil("apples");17echo $scanner->getRestUntilEnd();18echo $scanner->hasMore();19echo $scanner->hasMoreTokens();20echo $scanner->isAtEnd();21echo $scanner->isAtTokenEnd();22echo $scanner->isAtTokenStart();23echo $scanner->isAtStart();24echo $scanner->isNext("apples");25echo $scanner->isNextToken("apples");26echo $scanner->isNextWord("apples");27echo $scanner->isNextWordToken("apples");28echo $scanner->isNextWordToken("apples");29echo $scanner->isPrevious("I");30echo $scanner->isPreviousToken("I");31echo $scanner->isPreviousWord("I");32echo $scanner->isPreviousWordToken("I");33echo $scanner->isPreviousWordToken("I");34echo $scanner->peek();35echo $scanner->peek(2);36echo $scanner->peek(2, 3);37echo $scanner->peek(3, 2);38echo $scanner->peek(3, 2, 1);39echo $scanner->peek(3, 2, 1, 4);40echo $scanner->peek(3, 2, 1, 4, 5);41echo $scanner->peek(3, 2, 1, 4, 5, 6);42echo $scanner->peek(3, 2, 1, 4, 5, 6, 7);43echo $scanner->peek(3, 2, 1, 4, 5, 6, 7, 8);44echo $scanner->peek(3, 2, 1,

StringTokenScanner

Using AI Code Generation

1package com.mercury.test;2import java.io.File;3import java.io.IOException;4import org.apache.poi.ss.usermodel.Row;5import org.apache.poi.ss.usermodel.Sheet;6import org.apache.poi.ss.usermodel.Workbook;7import org.apache.poi.ss.usermodel.WorkbookFactory;8import org.apache.poi.xssf.usermodel.XSSFWorkbook;9import com.cucumber.common.StringTokenScanner;10import com.cucumber.common.TestData;11public class ReadDataFromExcel {12	public static void main(String[] args) throws IOException {13		StringTokenScanner stringTokenScanner = new StringTokenScanner();14		TestData testData = new TestData();15		Workbook workbook = new XSSFWorkbook();16		Sheet sheet = workbook.createSheet();17		Row row = sheet.createRow(0);18		File file = new File("C:\\Users\\Sushma\\Documents\

StringTokenScanner

Using AI Code Generation

1$myVar = $this->getVariable('myVar');2$this->iClickOnTheLink($myVar);3$myVar = $this->getVariable('myVar');4$this->iClickOnTheButton($myVar);5$myVar = $this->getVariable('myVar');6$this->iClickOnTheLink($myVar);7$myVar = $this->getVariable('myVar');8$this->iClickOnTheButton($myVar);9$myVar = $this->getVariable('myVar');10$this->iClickOnTheLink($myVar);11$myVar = $this->getVariable('myVar');12$this->iClickOnTheButton($myVar);13$myVar = $this->getVariable('myVar');14$this->iClickOnTheLink($myVar);

Blogs

Check out the latest blogs from LambdaTest on this topic:

How To Choose The Right Mobile App Testing Tools

Did you know that according to Statista, the number of smartphone users will reach 18.22 billion by 2025? Let’s face it, digital transformation is skyrocketing and will continue to do so. This swamps the mobile app development market with various options and gives rise to the need for the best mobile app testing tools

April 2020 Platform Updates: New Browser, Better Performance & Much Much More!

Howdy testers! If you’re reading this article I suggest you keep a diary & a pen handy because we’ve added numerous exciting features to our cross browser testing cloud and I am about to share them with you right away!

Starting & growing a QA Testing career

The QA testing career includes following an often long, winding road filled with fun, chaos, challenges, and complexity. Financially, the spectrum is broad and influenced by location, company type, company size, and the QA tester’s experience level. QA testing is a profitable, enjoyable, and thriving career choice.

Webinar: Building Selenium Automation Framework [Voices of Community]

Even though several frameworks are available in the market for automation testing, Selenium is one of the most renowned open-source frameworks used by experts due to its numerous features and benefits.

Test Managers in Agile – Creating the Right Culture for Your SQA Team

I was once asked at a testing summit, “How do you manage a QA team using scrum?” After some consideration, I realized it would make a good article, so here I am. Understand that the idea behind developing software in a scrum environment is for development teams to self-organize.

Automation Testing Tutorials

Learn to execute automation testing from scratch with LambdaTest Learning Hub. Right from setting up the prerequisites to run your first automation test, to following best practices and diving deeper into advanced test scenarios. LambdaTest Learning Hubs compile a list of step-by-step guides to help you be proficient with different test automation frameworks i.e. Selenium, Cypress, TestNG etc.