KaneAI - World’s First E2E Software Testing Agent.Signup for Private Beta Join TestMu Conference

How to use completion method of lang Package

Best Gauge code snippet using lang.completion

metric.go

Source:metric.go

1package completions2// # Overview3// The new code lives in `completions/metrics.go`.4//5// A *completion* is uniquely identified by the final state of the buffer if the completion were to be6// selected as well as the cursor position post-selection. This is tracked as a *targetBuffer*.7// We keep track of the targetBuffers from the last time completions were returned.8// Each shownInstance of a completion has a *rank*, which is where in the returned list of completions it was shown.9//10// We track metrics for selected completions.11// Selected completions are those which were likely selected from the completions dropdown by the user.12//	   * a completion is selected if the current buffer matches one of the targetBuffers from the latest set of completions returned.13//     * this may be a subset of completed completions.14//     * there may be false positives here: if the user fully types out the completion, the last seen completions will include a shownInstance15//       typically just before the final character, and if the user just types the final character, we still consider16//       the completion selected.17//18// For each tracked completion, we track its rank, length, and number of characters inserted.19// This allows us to do some post-processing & analysis of the data on segment (such as ignoring completions20// where the number of inserted characters is 1).21//22// # Assumptions23// 1. completions requests & buffer edits are received in order; based on my testing, this seems to be the case24//    (almost) always for Kite Local; in the rare case that this may not hold, I posit that the resulting metrics25//    blip is acceptable in return for cleaner code.26// 2. a *single* buffer edit event is sent when the user selects a completion from the dropdown; this isn't a hard27//    requirement, but the `NumSelected` value being useful depends on this. Again, I tested this on all editors28//    except vim, and it appears to be true.29//30// # Extensions31// We can easily extend this framework in various ways. An obvious augmentation would be to track the last32// shownInstance of a completion and ranks in previous shownInstances. This might be useful, since it's33// possible a completion isn't selected until its rank becomes sufficiently high. It might be sufficient to just34// additionally track the first shownInstance for this sort of analysis.35import (36	"bytes"37	"compress/gzip"38	"encoding/base64"39	"encoding/json"40	"fmt"41	"hash/fnv"42	"log"43	"math/rand"44	"strconv"45	"strings"46	"sync"47	"time"48	"unicode/utf8"49	"github.com/kiteco/kiteco/kite-go/client/internal/clienttelemetry"50	"github.com/kiteco/kiteco/kite-go/lang"51	"github.com/kiteco/kiteco/kite-go/lang/lexical/lexicalcomplete/lexicalproviders"52	"github.com/kiteco/kiteco/kite-go/lang/python/pythonscanner"53	"github.com/kiteco/kiteco/kite-go/response"54	"github.com/kiteco/kiteco/kite-golib/complete/data"55	"github.com/kiteco/kiteco/kite-golib/lexicalv0"56	"github.com/kiteco/kiteco/kite-golib/lexicalv0/lexer"57	"github.com/kiteco/kiteco/kite-golib/rollbar"58	"github.com/kiteco/kiteco/kite-golib/stringindex"59)60const (61	durationMsBucketSize = 10  // 10 ms buckets62	durationMsMaxBucket  = 150 // everything greater than 150ms is grouped into a single bucket63	shownSampleRate      = 0.164)65// CompletedInfo encapsulates a completion's rank, length, and number of characters inserted66// Note that we don't currently serialize individual instances of this field anywhere; we just send aggregations67// TODO: send individual completed/selected samples in an event that's not kite_status68type CompletedInfo struct {69	// some fields are json CamelCase to maintain backwards compatibility for consumers70	// TODO(naman) rename Source, Rank, Len, NumInserted71	Source      response.EditorCompletionSource72	Provider    data.ProviderName73	Rank        int74	Len         int75	NumInserted int76	// num chars/tokens inserted since last shown77	NumTokensInserted int `json:"num_tokens_inserted"`78}79// EngineMetrics contains counts of completion request metrics that are generated by the completions engine, for80// a particular language81type EngineMetrics struct {82	// Aggregated Metrics83	RequestsReceived          int84	RequestsAccepted          int85	RequestsAcceptedNewToken  int86	RequestsFulfilled         int87	RequestsFulfilledNewToken int88	// Unit Metrics89	AcceptedPerProvider          map[data.ProviderName]int90	AcceptedPerProviderNewToken  map[data.ProviderName]int91	FulfilledPerProvider         map[data.ProviderName]int92	FulfilledPerProviderNewToken map[data.ProviderName]int93}94func newEngineMetrics() EngineMetrics {95	return EngineMetrics{96		AcceptedPerProvider:          make(map[data.ProviderName]int),97		AcceptedPerProviderNewToken:  make(map[data.ProviderName]int),98		FulfilledPerProvider:         make(map[data.ProviderName]int),99		FulfilledPerProviderNewToken: make(map[data.ProviderName]int),100	}101}102func (r EngineMetrics) flatten(l lang.Language, out map[string]interface{}) {103	write := func(k string, val interface{}) {104		if val, ok := val.(int); ok && val == 0 {105			// save space by not writing 0 values106			return107		}108		out[k] = val109	}110	write(fmt.Sprintf("%s_completions_request_received", l.Name()), r.RequestsReceived)111	write(fmt.Sprintf("%s_completions_request_accepted", l.Name()), r.RequestsAccepted)112	write(fmt.Sprintf("%s_completions_starting_request_accepted", l.Name()), r.RequestsAcceptedNewToken)113	write(fmt.Sprintf("%s_completions_request_fulfilled", l.Name()), r.RequestsFulfilled)114	write(fmt.Sprintf("%s_completions_starting_request_fulfilled", l.Name()), r.RequestsFulfilledNewToken)115	for p, count := range r.AcceptedPerProvider {116		write(fmt.Sprintf("%s_%s_completions_request_accepted", l.Name(), p.String()), count)117	}118	for p, count := range r.AcceptedPerProviderNewToken {119		write(fmt.Sprintf("%s_%s_completions_starting_request_accepted", l.Name(), p.String()), count)120	}121	for p, count := range r.FulfilledPerProvider {122		write(fmt.Sprintf("%s_%s_completions_request_fulfilled", l.Name(), p.String()), count)123	}124	for p, count := range r.FulfilledPerProviderNewToken {125		write(fmt.Sprintf("%s_%s_completions_starting_request_fulfilled", l.Name(), p.String()), count)126	}127}128// GetEngineMetricsCallback returns a standard EngineMetricsCallback that increments the relevant counts in a Metrics object129func (m *Metrics) GetEngineMetricsCallback(isNewTokenInsertion bool) data.EngineMetricsCallback {130	return func(acceptedProviders map[data.ProviderName]struct{}, fulfilledProviders map[data.ProviderName]struct{}) {131		m.m.Lock()132		defer m.m.Unlock()133		accepted := false134		for p := range acceptedProviders {135			accepted = true136			m.stats.EngineMetrics.AcceptedPerProvider[p]++137			if isNewTokenInsertion {138				m.stats.EngineMetrics.AcceptedPerProviderNewToken[p]++139			}140		}141		if accepted {142			m.stats.EngineMetrics.RequestsAccepted++143			if isNewTokenInsertion {144				m.stats.EngineMetrics.RequestsAcceptedNewToken++145			}146		}147		fulfilled := false148		for p := range fulfilledProviders {149			fulfilled = true150			m.stats.EngineMetrics.FulfilledPerProvider[p]++151			if isNewTokenInsertion {152				m.stats.EngineMetrics.FulfilledPerProviderNewToken[p]++153			}154		}155		if fulfilled {156			m.stats.EngineMetrics.RequestsFulfilled++157			if isNewTokenInsertion {158				m.stats.EngineMetrics.RequestsFulfilledNewToken++159			}160		}161	}162}163// ReportedCompletionMetrics contains counts of completion metrics that the editors have reported to kited for one164// language165type ReportedCompletionMetrics struct {166	TotalSelected      int167	TotalWordsInserted int168	SelectedPerEditor        map[data.Editor]int169	SelectedPerProvider      map[data.ProviderName]int170	InsertedWordsPerProvider map[data.ProviderName]int171}172func newReportedCompletionMetrics() ReportedCompletionMetrics {173	return ReportedCompletionMetrics{174		SelectedPerEditor:        make(map[data.Editor]int),175		SelectedPerProvider:      make(map[data.ProviderName]int),176		InsertedWordsPerProvider: make(map[data.ProviderName]int),177	}178}179func (r ReportedCompletionMetrics) flatten(l lang.Language, out map[string]interface{}) {180	write := func(k string, val interface{}) {181		if val, ok := val.(int); ok && val == 0 {182			// save space by not writing 0 values183			return184		}185		out[k] = val186	}187	write(fmt.Sprintf("%s_completions_selected", l.Name()), r.TotalSelected)188	write(fmt.Sprintf("%s_completions_words_inserted", l.Name()), r.TotalWordsInserted)189	for editor, count := range r.SelectedPerEditor {190		write(fmt.Sprintf("%s_%s_completions_selected", editor.String(), l.Name()), count)191	}192	for p, count := range r.SelectedPerProvider {193		write(fmt.Sprintf("%s_%s_completions_selected", l.Name(), p.String()), count)194	}195	for p, count := range r.InsertedWordsPerProvider {196		write(fmt.Sprintf("%s_%s_completions_words_inserted", l.Name(), p.String()), count)197	}198}199// RequestCounts contains counts of requests made for completions200type RequestCounts struct {201	Total      int // number of total requests202	Expected   int // number of expected requests203	Unexpected int // number of unexpected requests204	Error      int // number of requests that resulted in an error205}206// SourceBreakdown describes a breakdown of counts by completions source.207type SourceBreakdown map[response.EditorCompletionSource]int208// Snapshot captures a snapshot of the statistics collected by this metric209type Snapshot struct {210	Requested    RequestCounts // breakdown of number of requests made for a changed buffer state211	RequestedRaw RequestCounts // breakdown of number of requests made, without buffer state deduplication212	Triggered          int // number of times completions were triggered for a changed buffer state213	Shown              int // number of unique completions that were shown214	AtLeastOneShown    int // number of times at least one completion was shown215	AtLeastOneNewShown int // number of times at least one new completion was shown216	NumSelected        int // total number of selected completions217	// number of unique completions shown by source218	ShownBySource SourceBreakdown219	// number of times at least one completion was shown, by source220	AtLeastOneShownBySource SourceBreakdown221	// number of times at least one new completion was shown, by source222	AtLeastOneNewShownBySource SourceBreakdown223	// SelectedBySource contains counts of selected completions by source224	SelectedBySource SourceBreakdown225	// Selected2BySource contains counts of selected completions, where at least 2 characters were inserted226	// during the selection, by source227	Selected2BySource SourceBreakdown228	DurationHistogram map[int64]uint229	// MatchedByProvider contains counts of completions that were matched (i.e. we infer they were selected),230	// by Provider231	MatchedByProvider map[data.ProviderName]int232	// EngineMetrics contains counts of completion request metrics that are kept track of within kited233	EngineMetrics EngineMetrics234	// ReportedCounts contains counts of completion metrics as they have been reported by editors235	ReportedCounts ReportedCompletionMetrics236}237func newSnapshot() Snapshot {238	return Snapshot{239		ShownBySource:              make(SourceBreakdown),240		AtLeastOneShownBySource:    make(SourceBreakdown),241		AtLeastOneNewShownBySource: make(SourceBreakdown),242		SelectedBySource:           make(SourceBreakdown),243		Selected2BySource:          make(SourceBreakdown),244		DurationHistogram:          make(map[int64]uint, durationMsMaxBucket/durationMsBucketSize+1),245		MatchedByProvider:          make(map[data.ProviderName]int),246		EngineMetrics:              newEngineMetrics(),247		ReportedCounts:             newReportedCompletionMetrics(),248	}249}250func (s *Snapshot) addSelected(new CompletedInfo) {251	s.NumSelected++252	s.SelectedBySource[new.Source]++253	s.MatchedByProvider[new.Provider]++254	if new.NumInserted >= 2 {255		s.Selected2BySource[new.Source]++256	}257}258func (s Snapshot) flatten(l lang.Language, out map[string]interface{}) {259	var prefix string260	switch l {261	case lang.Python:262		// This is for backwards compatibility reasons (python came first)263		prefix = ""264	default:265		prefix = l.Name() + "_"266	}267	write := func(k string, val interface{}) {268		if val, ok := val.(int); ok && val == 0 {269			// save space by not writing 0 values270			return271		}272		out[k] = val273	}274	write(prefix+"completions_requested", s.Requested.Total)275	write(prefix+"completions_requested_expected", s.Requested.Expected)276	write(prefix+"completions_requested_unexpected", s.Requested.Unexpected)277	write(prefix+"completions_requested_error", s.Requested.Error)278	write(prefix+"completions_requested_raw", s.RequestedRaw.Total)279	write(prefix+"completions_requested_expected_raw", s.RequestedRaw.Expected)280	write(prefix+"completions_requested_unexpected_raw", s.RequestedRaw.Unexpected)281	write(prefix+"completions_requested_error_raw", s.RequestedRaw.Error)282	write(prefix+"completions_triggered", s.Triggered)283	write(prefix+"completions_shown", s.Shown)284	write(prefix+"completions_at_least_one_shown", s.AtLeastOneShown)285	write(prefix+"completions_at_least_one_new_shown", s.AtLeastOneNewShown)286	// Old version287	write(prefix+"completions_num_selected", s.NumSelected)288	// New version289	write(fmt.Sprintf("%s_completions_matched", l.Name()), s.NumSelected)290	for p, count := range s.MatchedByProvider {291		write(fmt.Sprintf("%s_%s_completions_matched", l.Name(), p.String()), count)292	}293	write(prefix+"completions_selected_by_source", s.SelectedBySource)294	write(prefix+"completions_selected_2_by_source", s.Selected2BySource)295	write(prefix+"completions_shown_by_source", s.ShownBySource)296	write(prefix+"completions_at_least_one_shown_by_source", s.AtLeastOneShownBySource)297	write(prefix+"completions_at_least_one_new_shown_by_source", s.AtLeastOneNewShownBySource)298	histogram := make(map[string]uint)299	for msLower, count := range s.DurationHistogram {300		msLowerStr := strconv.Itoa(int(msLower))301		var key string302		if msLower == durationMsMaxBucket {303			key = msLowerStr + "_"304		} else {305			msUpperStr := strconv.Itoa(int(msLower + durationMsBucketSize - 1))306			key = msLowerStr + "_" + msUpperStr307		}308		histogram[key] = count309	}310	write(prefix+"completions_duration_hist", histogram)311	s.EngineMetrics.flatten(l, out)312	s.ReportedCounts.flatten(l, out)313}314// MetricsByLang tracks metric sharded by language315type MetricsByLang struct {316	m sync.Map317}318// NewMetrics returns a new MetricsByLang319func NewMetrics() *MetricsByLang {320	return &MetricsByLang{}321}322// Get gets the metrics object for a given language323func (ms *MetricsByLang) Get(l lang.Language) *Metrics {324	if ms == nil {325		return nil326	}327	shard, ok := ms.m.Load(l)328	if !ok {329		m := &Metrics{330			s:     newBufferState(),331			stats: newSnapshot(),332			lang:  l,333		}334		shard, _ = ms.m.LoadOrStore(l, m)335	}336	return shard.(*Metrics)337}338// ReadAndFlatten metrics to send339func (ms *MetricsByLang) ReadAndFlatten(clear bool, out map[string]interface{}) map[string]interface{} {340	if out == nil {341		out = make(map[string]interface{})342	}343	ms.m.Range(func(k, v interface{}) bool {344		l := k.(lang.Language)345		m := v.(*Metrics)346		m.read(clear).flatten(l, out)347		return true348	})349	return out350}351// Flush flushes the metrics for each language352func (ms *MetricsByLang) Flush() {353	ms.m.Range(func(k, v interface{}) bool {354		m := v.(*Metrics)355		m.flush()356		return true357	})358}359// Metrics records the number of completions used, out of those suggested by kite.360type Metrics struct {361	m          sync.Mutex362	s          *bufferState363	stats      Snapshot364	lang       lang.Language365	cLock      sync.Mutex366	complStats []CompletionStat367}368// Read returns the current count369func (m *Metrics) read(clear bool) Snapshot {370	m.m.Lock()371	defer m.m.Unlock()372	out := m.stats373	if clear {374		m.stats = newSnapshot()375		m.s.shown = make(map[targetBuffer]struct{})376		return out377	}378	// copy slices and maps to avoid data races379	copyBreakdown := func(src SourceBreakdown) SourceBreakdown {380		ret := make(SourceBreakdown, len(src))381		for k, v := range src {382			ret[k] = v383		}384		return ret385	}386	out.ShownBySource = copyBreakdown(out.ShownBySource)387	out.AtLeastOneShownBySource = copyBreakdown(out.AtLeastOneShownBySource)388	out.AtLeastOneNewShownBySource = copyBreakdown(out.AtLeastOneNewShownBySource)389	out.SelectedBySource = copyBreakdown(out.SelectedBySource)390	out.Selected2BySource = copyBreakdown(out.Selected2BySource)391	return out392}393// Requested is called when completions are requested by the user394func (m *Metrics) Requested() {395	if m == nil {396		return397	}398	m.m.Lock()399	defer m.m.Unlock()400	m.stats.RequestedRaw.Total++401	if !m.s.eventsSeen.requested {402		m.stats.Requested.Total++403	}404	m.s.eventsSeen.requested = true405	m.stats.EngineMetrics.RequestsReceived++406}407// Expected is called when a response is received to record whether or not the original request was expected408func (m *Metrics) Expected(expected bool) {409	if m == nil {410		return411	}412	m.m.Lock()413	defer m.m.Unlock()414	if expected {415		m.stats.RequestedRaw.Expected++416		if !m.s.eventsSeen.requestedExpected {417			m.stats.Requested.Expected++418		}419		m.s.eventsSeen.requestedExpected = true420	} else {421		m.stats.RequestedRaw.Unexpected++422		if !m.s.eventsSeen.requestedUnexpected {423			m.stats.Requested.Unexpected++424		}425		m.s.eventsSeen.requestedUnexpected = true426	}427}428// Errored is called when there was an error obtaining completions429func (m *Metrics) Errored() {430	if m == nil {431		return432	}433	m.m.Lock()434	defer m.m.Unlock()435	m.stats.RequestedRaw.Error++436	if !m.s.eventsSeen.requestedError {437		m.stats.Requested.Error++438	}439	m.s.eventsSeen.requestedError = true440}441// ReturnedCompat is a backwards compatible wrapper around Returned442func (m *Metrics) ReturnedCompat(text string, cursor int, legacy []response.EditorCompletion, start time.Time) {443	if m == nil {444		return445	}446	var compls []data.NRCompletion447	for _, compl := range legacy {448		start := legacyComplStart(text, cursor, compl.Insert)449		rc := data.RCompletion{450			Completion: data.Completion{451				Snippet: data.Snippet{Text: compl.Insert},452				Replace: data.Selection{Begin: start, End: cursor},453			},454			Source: compl.Source,455		}456		compls = append(compls, data.NRCompletion{RCompletion: rc})457	}458	resp := data.NewAPIResponse(data.APIRequest{459		SelectedBuffer: data.NewBuffer(text).Select(data.Cursor(cursor)),460	})461	resp.Completions = compls462	m.Returned(resp, start)463}464// Returned is called when completions are sent to the user465func (m *Metrics) Returned(resp data.APIResponse, start time.Time) {466	if m == nil {467		return468	}469	resp.EncodeOffsets(stringindex.Native)470	sb := resp.Request.SelectedBuffer471	compls := resp.Completions472	duration := time.Since(start)473	m.m.Lock()474	defer m.m.Unlock()475	if len(compls) > 0 {476		bucket := int64(duration/time.Millisecond) / durationMsBucketSize * durationMsBucketSize477		if bucket > durationMsMaxBucket {478			bucket = durationMsMaxBucket479		}480		m.stats.DurationHistogram[bucket]++481	}482	if sb.End > sb.Begin {483		return484	}485	cursor := sb.Begin486	text := sb.Text()487	if cursor < 0 || cursor > len(text) {488		rollbar.Error(fmt.Errorf("cursor out of bounds"))489		return490	}491	m.updateLocked(buffer{data.NewBuffer(text).Select(data.Cursor(cursor))})492	if !m.s.eventsSeen.triggered {493		m.stats.Triggered++494	}495	m.s.eventsSeen.triggered = true496	sources := make(map[response.EditorCompletionSource]struct{})497	for _, c := range compls {498		sources[c.Source] = struct{}{}499	}500	var totalAtLeastOne int501	for source := range sources {502		if _, found := m.s.eventsSeen.atLeastOneShown[source]; !found {503			m.stats.AtLeastOneShownBySource[source]++504			totalAtLeastOne++505		}506		m.s.eventsSeen.atLeastOneShown[source] = true507	}508	if totalAtLeastOne > 0 {509		m.stats.AtLeastOneShown++510	}511	var shown int512	newSources := make(map[response.EditorCompletionSource]struct{})513	for rank, compl := range compls {514		if isNew, inst := m.s.add(compl.RCompletion, rank); isNew {515			m.recordStat(inst, completionShown)516			shown++517			newSources[compl.Source] = struct{}{}518			m.stats.ShownBySource[compl.Source]++519		}520	}521	m.stats.Shown += shown522	if shown > 0 {523		m.stats.AtLeastOneNewShown++524	}525	for source := range newSources {526		m.stats.AtLeastOneNewShownBySource[source]++527	}528}529// BufferEdited is called when we receive the contents of the buffer,530// NOTE: `cursor` must be the byte offset (from the beginning of the `text`) to the current cursor position.531func (m *Metrics) BufferEdited(text string, cursor int) {532	if m == nil {533		return534	}535	m.m.Lock()536	defer m.m.Unlock()537	if cursor < 0 || cursor > len(text) {538		rollbar.Error(fmt.Errorf("cursor out of bounds"))539		return540	}541	m.updateLocked(buffer{data.NewBuffer(text).Select(data.Cursor(cursor))})542}543// CompletionSelectedEvent is a completion selection event sent from editors to kited544type CompletionSelectedEvent struct {545	Editor     data.Editor      `json:"editor"`546	Language   string           `json:"language"`547	Completion data.RCompletion `json:"completion"`548}549// CompletionSelected is called when we receive a completion selected event550func (m *Metrics) CompletionSelected(cs CompletionSelectedEvent) {551	m.m.Lock()552	defer m.m.Unlock()553	compl := cs.Completion554	editor := cs.Editor555	m.stats.ReportedCounts.TotalSelected++556	m.stats.ReportedCounts.SelectedPerProvider[compl.Provider]++557	m.stats.ReportedCounts.SelectedPerEditor[editor]++558	wordCount := numWords(compl.Snippet.Text, m.lang)559	m.stats.ReportedCounts.TotalWordsInserted += wordCount560	m.stats.ReportedCounts.InsertedWordsPerProvider[compl.Provider] += wordCount561}562// -563func legacyComplStart(text string, cursor int, complText string) int {564	testPos := cursor565	start := testPos566	for start > 0 {567		r, sz := utf8.DecodeLastRuneInString(text[:testPos])568		if r == utf8.RuneError {569			// TODO(naman) track this in metrics?570			log.Printf("could not decode rune in completionsmetrics")571			return cursor572		}573		testPos -= sz574		testSuffix := text[testPos:cursor]575		idx := strings.Index(complText, testSuffix)576		if idx < 0 || testPos < idx { // TODO(naman) why do we need the second disjunct? leave a comment here577			break578		}579		// the first match of testSuffix is at position idx, so if there is a valid overlap, it must start at testPos-idx580		testPos -= idx581		testSuffix = text[testPos:cursor]582		if strings.HasPrefix(complText, testSuffix) {583			start = testPos584		}585	}586	return start587}588func (m *Metrics) updateLocked(b buffer) {589	completion := m.s.update(b)590	if completion == nil {591		return592	}593	log.Printf("%s completion selection inferred", m.lang.Name())594	m.recordStat(completion, completionSelected)595	inserted := completion.toInsert()596	cs := CompletedInfo{597		Rank:        completion.rank,598		Len:         len(completion.text),599		NumInserted: len(inserted),600		Source:      completion.source,601		Provider:    completion.provider,602		NumTokensInserted: numTokens(inserted, m.lang),603		// TODO(naman) NumTokensFirstShown: c.numTokensFirstShown,604	}605	m.stats.addSelected(cs)606}607// --608type buffer struct {609	data.SelectedBuffer // we assume the Selection is just a fixed cursor for now610}611// targetBuffer encapsulates a "logical" completion, represented by the final "target" buffer state resulting from completing a completion612type targetBuffer struct {613	hash   uint64 // hash of entire buffer, post-completion614	cursor int    // cursor position, post-completion615}616// shownInstance encapsulates information about how & when a completion was shown617type shownInstance struct {618	// text of the completion, potentially including already typed (case-insensitive) characters619	text string620	// source is where the completion was generated621	source response.EditorCompletionSource622	// provider that provided the completion623	provider data.ProviderName624	// rank in the list of completions returned to the editor625	rank int626	// cursor position when the completion was shown627	cursor int628	// start position of the completion text in the buffer629	start int630	// end position of the completion text in the buffer631	end int632	// replacedText is the text being replaced by the completion633	replacedText string634	// placeholderChars is the total sum of chars in the snippet placeholders635	placeholderChars int636	// lexical indicates whether this represents a lexical completion637	lexical bool638	// lexicalMetrics contains lexical-specific metrics639	lexicalMetrics *lexicalproviders.LexicalMetrics640}641type completionStatus string642type completionType string643const (644	invalidStatus completionStatus = "invalid"645	validStatus   completionStatus = "valid"646	completionShown    completionType = "shown"647	completionSelected completionType = "selected"648)649func (i shownInstance) toInsert() string {650	progress := i.progress()651	if progress > len(i.text) {652		return ""653	}654	return i.text[progress:]655}656func (i shownInstance) progress() int {657	return i.cursor - i.start658}659func (b buffer) shownInstance(compl data.RCompletion, rank int) *shownInstance {660	phChars := 0661	for _, ph := range compl.Snippet.Placeholders() {662		phChars += ph.Len()663	}664	instance := &shownInstance{665		text:             compl.Snippet.Text,666		source:           compl.Source,667		provider:         compl.Provider,668		rank:             rank,669		cursor:           b.Selection.Begin,670		start:            compl.Replace.Begin,671		end:              compl.Replace.End,672		replacedText:     b.TextAt(compl.Replace),673		placeholderChars: phChars,674	}675	if metrics, ok := compl.Metrics.(*lexicalproviders.LexicalMetrics); ok {676		instance.lexical = true677		instance.lexicalMetrics = metrics678	}679	return instance680}681func (b buffer) targetBuffer(i *shownInstance) targetBuffer {682	text := b.TextAt(data.Selection{End: i.start}) + i.text + b.TextAt(data.Selection{Begin: i.end, End: b.Buffer.Len()})683	return targetBuffer{684		hash: hash(text),685	}686}687func (b buffer) status(target targetBuffer, last *shownInstance) completionStatus {688	if b.Selection.Begin < last.cursor {689		// cursor moved backwards690		return invalidStatus691	}692	text := b.TextAt(data.Selection{End: last.start}) + last.text + b.TextAt(data.Selection{Begin: last.end, End: b.Buffer.Len()})693	if target.hash != hash(text) {694		// contents of buffer before cursor changed, accounting for difference695		return invalidStatus696	}697	return validStatus698}699// - tracking700// max number of completion stats to record before sending:701// as of 2019-11-13, the completion selected(2) per shown rate is ~15%,702// so we'd send an event every 150/.15 = 1000 times completions are shown.703//704// Ammendment: This was cut in half to 75 after the addition of `Lexical` and705// `LexicalMetrics` to CompletionStat, as they roughly doubled the size of the object706var maxStats = 75707// CompletionStat encapsulates the information for a completion that is shown or selected708type CompletionStat struct {709	Language       lang.Language  `json:"language"`710	CompletionType completionType `json:"completion_type"`711	Source         string         `json:"source"`712	Rank           int            `json:"rank"`713	// counts for insertion text (stripped of prefix/suffix matches)714	NumCharsInserted  int `json:"num_chars_inserted"`715	NumTokensInserted int `json:"num_tokens_inserted"` // includes EOF716	// counts for buffer text being replaced717	NumCharsReplaced  int `json:"num_chars_replaced"`718	NumTokensReplaced int `json:"num_tokens_replaced"` // includes EOF719	// total number of characters in the placeholders720	NumPlaceholderChars int `json:"num_placeholder_chars"`721	Lexical        bool                             `json:"is_lexical"`722	LexicalMetrics *lexicalproviders.LexicalMetrics `json:"lexical_metrics,omitempty"` // omitempty saves some space723}724type eventsSeen struct {725	triggered           bool726	atLeastOneShown     map[response.EditorCompletionSource]bool727	requested           bool728	requestedExpected   bool729	requestedUnexpected bool730	requestedError      bool731}732type bufferState struct {733	lastSeen   map[targetBuffer]*shownInstance // completions seen since update was called734	shown      map[targetBuffer]struct{}       // all completions shown for the current snapshot735	buffer     buffer736	eventsSeen eventsSeen // the events that have been seen between buffer updates737}738func (s *bufferState) update(b buffer) *shownInstance {739	if b == s.buffer {740		return nil741	}742	s.buffer = b743	s.eventsSeen = eventsSeen{744		atLeastOneShown: make(map[response.EditorCompletionSource]bool),745	}746	defer func() {747		s.lastSeen = make(map[targetBuffer]*shownInstance)748	}()749	targetBuffer := targetBuffer{750		hash: hash(b.Text()),751	}752	if inst, exists := s.lastSeen[targetBuffer]; exists {753		return inst754	}755	return nil756}757// add returns true if the provided shownInstance corresponds to a newly seen & valid targetBuffer;758// otherwise it returns false.759func (s *bufferState) add(compl data.RCompletion, rank int) (bool, *shownInstance) {760	inst := s.buffer.shownInstance(compl, rank)761	target := s.buffer.targetBuffer(inst)762	switch s.buffer.status(target, inst) {763	case invalidStatus:764		rollbar.Error(fmt.Errorf("completion shown with invalid status"))765		return false, nil766	}767	_, exists := s.shown[target]768	s.shown[target] = struct{}{}769	s.lastSeen[target] = inst770	return !exists, inst771}772func (m *Metrics) recordStat(inst *shownInstance, cType completionType) {773	if cType == completionShown {774		if !inst.lexical {775			return776		}777		if rand.Float64() > shownSampleRate {778			return779		}780	}781	m.cLock.Lock()782	defer m.cLock.Unlock()783	stat := CompletionStat{784		Language:            m.lang,785		CompletionType:      cType,786		Source:              string(inst.source),787		Rank:                inst.rank,788		NumCharsInserted:    len(inst.toInsert()),789		NumCharsReplaced:    len(inst.replacedText),790		NumTokensInserted:   numTokens(inst.toInsert(), m.lang),791		NumTokensReplaced:   numTokens(inst.replacedText, m.lang),792		NumPlaceholderChars: inst.placeholderChars,793		Lexical:             inst.lexical,794		LexicalMetrics:      inst.lexicalMetrics,795	}796	m.complStats = append(m.complStats, stat)797	if len(m.complStats) > maxStats {798		m.flush()799	}800}801func (m *Metrics) flush() {802	clienttelemetry.KiteTelemetry("Completion Stats", map[string]interface{}{803		"completion_stats": m.encodeStatsLocked(),804	})805	m.complStats = nil806}807func (m *Metrics) encodeStatsLocked() string {808	var buf bytes.Buffer809	gzw := gzip.NewWriter(&buf)810	json.NewEncoder(gzw).Encode(&m.complStats)811	gzw.Close()812	return base64.StdEncoding.EncodeToString(buf.Bytes())813}814func newBufferState() *bufferState {815	return &bufferState{816		lastSeen: make(map[targetBuffer]*shownInstance),817		shown:    make(map[targetBuffer]struct{}),818	}819}820// --821// compute a fast non-cryptographic hash822func hash(strs ...string) uint64 {823	h := fnv.New64()824	for _, s := range strs {825		h.Write([]byte(s))826	}827	return h.Sum64()828}829func numTokens(s string, l lang.Language) int {830	if l == lang.Python {831		cnt, _ := pythonscanner.Count([]byte(s), pythonscanner.DefaultOptions)832		return cnt833	}834	langLexer, err := lexicalv0.NewLexerForMetrics(l)835	if err != nil {836		return 0837	}838	toks, err := langLexer.Lex([]byte(s))839	if err != nil {840		return 0841	}842	var cnt int843	for _, tok := range toks {844		// TODO(tarak): This filters out the EOF token and semicolon. The semicolon is primarily to845		// remove the automatically inserted semicolons at the end of JS/GO lines that are not846		// rendered, but will also remove them from for loops, etc. We can iterate on this if we like.847		if langLexer.IsType(lexer.EOF, tok) || langLexer.IsType(lexer.SEMICOLON, tok) {848			continue849		}850		cnt++851	}852	return cnt853}854// Counts the number of words in a completion, where words are tokens that are identifiers, keywords, literals, and855// similar interesting tokens. These words are not the same as pythonscanner.Word.856func numWords(s string, l lang.Language) int {857	langLexer, err := lexicalv0.NewLexerForMetrics(l)858	if err != nil {859		return 0860	}861	tokens, err := langLexer.Lex([]byte(s))862	if err != nil {863		return 0864	}865	var wordCount int866	for _, token := range tokens {867		if langLexer.IsType(lexer.IDENT, token) || langLexer.IsType(lexer.KEYWORD, token) || langLexer.IsType(lexer.LITERAL, token) || langLexer.IsType(lexer.IMPORT, token) {868			wordCount++...

completion.go

Source:completion.go

1package lsp2import (3	"github.com/hashicorp/hcl-lang/lang"4	"github.com/hashicorp/terraform-ls/internal/mdplain"5	lsp "github.com/hashicorp/terraform-ls/internal/protocol"6)7func ToCompletionList(candidates lang.Candidates, caps lsp.TextDocumentClientCapabilities) lsp.CompletionList {8	list := lsp.CompletionList{9		Items:        make([]lsp.CompletionItem, len(candidates.List)),10		IsIncomplete: !candidates.IsComplete,11	}12	for i, c := range candidates.List {13		list.Items[i] = toCompletionItem(c, caps.Completion)14	}15	return list16}17func toCompletionItem(candidate lang.Candidate, caps lsp.CompletionClientCapabilities) lsp.CompletionItem {18	snippetSupport := caps.CompletionItem.SnippetSupport19	doc := candidate.Description.Value20	// TODO: Revisit when MarkupContent is allowed as Documentation21	// https://github.com/golang/tools/blob/4783bc9b/internal/lsp/protocol/tsprotocol.go#L75322	doc = mdplain.Clean(doc)23	var kind lsp.CompletionItemKind24	switch candidate.Kind {25	case lang.AttributeCandidateKind:26		kind = lsp.PropertyCompletion27	case lang.BlockCandidateKind:28		kind = lsp.ClassCompletion29	case lang.LabelCandidateKind:30		kind = lsp.FieldCompletion31	case lang.BoolCandidateKind:32		kind = lsp.EnumMemberCompletion33	case lang.StringCandidateKind:34		kind = lsp.TextCompletion35	case lang.NumberCandidateKind:36		kind = lsp.ValueCompletion37	case lang.KeywordCandidateKind:38		kind = lsp.KeywordCompletion39	case lang.ListCandidateKind, lang.SetCandidateKind, lang.TupleCandidateKind:40		kind = lsp.EnumCompletion41	case lang.MapCandidateKind, lang.ObjectCandidateKind:42		kind = lsp.StructCompletion43	case lang.TraversalCandidateKind:44		kind = lsp.VariableCompletion45	}46	// TODO: Omit item which uses kind unsupported by the client47	var cmd *lsp.Command48	if candidate.TriggerSuggest && snippetSupport {49		cmd = &lsp.Command{50			Command: "editor.action.triggerSuggest",51			Title:   "Suggest",52		}53	}54	item := lsp.CompletionItem{55		Label:               candidate.Label,56		Kind:                kind,57		InsertTextFormat:    insertTextFormat(snippetSupport),58		Detail:              candidate.Detail,59		Documentation:       doc,60		TextEdit:            textEdit(candidate.TextEdit, snippetSupport),61		Command:             cmd,62		AdditionalTextEdits: textEdits(candidate.AdditionalTextEdits, snippetSupport),63	}64	if caps.CompletionItem.DeprecatedSupport {65		item.Deprecated = candidate.IsDeprecated66	}67	if tagSliceContains(caps.CompletionItem.TagSupport.ValueSet,68		lsp.ComplDeprecated) && candidate.IsDeprecated {69		item.Tags = []lsp.CompletionItemTag{70			lsp.ComplDeprecated,71		}72	}73	return item74}75func tagSliceContains(supported []lsp.CompletionItemTag, tag lsp.CompletionItemTag) bool {76	for _, item := range supported {77		if item == tag {78			return true79		}80	}81	return false82}...

completion

Using AI Code Generation

1import (2func main() {3	lang.Completion()4}5import (6func main() {7	lang.Completion()8}9import (10func main() {11	lang.Completion()12}13import (14func main() {15	lang.Completion()16}17import (18func main() {19	lang.Completion()20}21import (22func main() {23	lang.Completion()24}25import (26func main() {27	lang.Completion()28}29import (30func main() {31	lang.Completion()32}33import (34func main() {35	lang.Completion()36}37import (38func main() {39	lang.Completion()40}41import (42func main() {43	lang.Completion()44}45import (46func main() {47	lang.Completion()48}49import (50func main() {

completion

Using AI Code Generation

1import "fmt"2import "strings"3func main() {4    fmt.Println(strings.HasPrefix(lang, "G"))5    fmt.Println(strings.HasSuffix(lang, "o"))6    fmt.Println(strings.Contains(lang, "o"))7}8import "fmt"9import "strings"10func main() {11    fmt.Println(strings.HasPrefix(lang, "g"))12    fmt.Println(strings.HasSuffix(lang, "O"))13    fmt.Println(strings.Contains(lang, "O"))14}15import "fmt"16import "strings"17func main() {18    fmt.Println(strings.HasPrefix(lang, "G"))19    fmt.Println(strings.HasSuffix(lang, "o"))20    fmt.Println(strings.Contains(lang, "O"))21}22import "fmt"23import "strings"24func main() {25    fmt.Println(strings.HasPrefix(lang, "g"))26    fmt.Println(strings.HasSuffix(lang, "O"))27    fmt.Println(strings.Contains(lang, "o"))28}29import "fmt"30import "strings"31func main() {32    fmt.Println(strings.HasPrefix(lang, "G"))33    fmt.Println(strings.HasSuffix(lang, "o"))34    fmt.Println(strings.Contains(lang, "o"))35}36import "fmt"37import "strings"38func main() {39    fmt.Println(strings.HasPrefix(lang, "g"))40    fmt.Println(strings.HasSuffix(lang, "O"))41    fmt.Println(strings.Contains(lang, "O"))42}43import "fmt"44import "strings"45func main() {46    fmt.Println(strings.HasPrefix(lang

completion

Using AI Code Generation

1import "fmt"2func main() {3	fmt.Println("Hello, World!")4}5import "fmt"6func main() {7	fmt.Println("Hello, World!")8	fmt.Println("Hello, World!")9}10import "fmt"11func main() {12	fmt.Println("Hello, World!")13	fmt.Println("Hello, World!")14	fmt.Println("Hello, World!")15}16import "fmt"17func main() {18	fmt.Println("Hello, World!")19	fmt.Println("Hello, World!")20	fmt.Println("Hello, World!")21	fmt.Println("Hello, World!")22}23import "fmt"24func main() {25	fmt.Println("Hello, World!")26	fmt.Println("Hello, World!")27	fmt.Println("Hello, World!")28	fmt.Println("Hello, World!")29	fmt.Println("Hello, World!")30}31import "fmt"32func main() {33	fmt.Println("Hello, World!"

completion

Using AI Code Generation

1import (2func main() {3	lang.Completion()4	fmt.Println(lang.Version())5}6import "fmt"7func Completion() {8	fmt.Println("Completion of lang package")9}10func Version() string {11}12import "testing"13func TestCompletion(t *testing.T) {14	Completion()15}16import (17func main() {18	lang.Completion()19	fmt.Println(lang.Version())20}21import "fmt"22func Completion() {23	fmt.Println("Completion of lang package")24}25func Version() string {26}27import "testing"28func TestCompletion(t *testing.T) {29	Completion()30}

completion

Using AI Code Generation

1import "fmt"2func main() {3    fmt.Scanln(&a)4    fmt.Println(a)5    fmt.Println("Hello World")6}7import "fmt"8func main() {9    fmt.Scanln(&a)10    fmt.Println(a)11    fmt.Println("Hello World")12}13import "fmt"14func main() {15    fmt.Scanln(&a)16    fmt.Println(a)17    fmt.Println("Hello World")18}19import "fmt"20func main() {21    fmt.Scanln(&a)22    fmt.Println(a)23    fmt.Println("Hello World")24}25import "fmt"26func main() {27    fmt.Scanln(&a)28    fmt.Println(a)29    fmt.Println("Hello World")30}31import "fmt"32func main() {33    fmt.Scanln(&a)34    fmt.Println(a)35    fmt.Println("Hello World")36}37import "fmt"38func main() {39    fmt.Scanln(&a)40    fmt.Println(a)41    fmt.Println("Hello World")42}43import "fmt"44func main() {45    fmt.Scanln(&a)46    fmt.Println(a)47    fmt.Println("Hello World")48}49import "fmt"50func main() {51    fmt.Scanln(&a)52    fmt.Println(a)53    fmt.Println("Hello World")54}55import "fmt"56func main() {57    fmt.Scanln(&

completion

Using AI Code Generation

1import (2func main() {3	fmt.Println(strings.Contains(str, "World"))4}5import (6func main() {7	fmt.Println(strings.ContainsAny(str, "World"))8}9import (10func main() {11	fmt.Println(strings.ContainsRune(str, 'W'))12}13import (14func main() {15	fmt.Println(strings.Count(str, "l"))16}17import (18func main() {19	fmt.Println(strings.EqualFold(str, "hello world"))20}21import (22func main() {

completion

Using AI Code Generation

1import (2func main() {3	lang.Completion()4	fmt.Println("Hello World!")5}6import "fmt"7func Completion() {8	fmt.Println("Go is a compiled, statically-typed language developed at Google in 2007 by Robert Griesemer, Rob Pike, and Ken Thompson. It is syntactically similar to C, but with memory safety, garbage collection, structural typing, and CSP-style concurrency.")9}

completion

Using AI Code Generation

1import (2func main() {3	lang.Completion("Go")4	fmt.Println(lang.Version())5}6import (7func main() {8	fmt.Println(lang.Completion("Go"))9}10import (11func main() {12	fmt.Println(lang.Completion("Go"))13}14import (15func main() {16	fmt.Println(lang.Completion("Go"))17}18import (19func main() {20	fmt.Println(lang.Completion("Go"))21}22import (23func main() {24	fmt.Println(lang.Completion("Go"))

Automation Testing Tutorials

Learn to execute automation testing from scratch with LambdaTest Learning Hub. Right from setting up the prerequisites to run your first automation test, to following best practices and diving deeper into advanced test scenarios. LambdaTest Learning Hubs compile a list of step-by-step guides to help you be proficient with different test automation frameworks i.e. Selenium, Cypress, TestNG etc.