How to use extract_bucket_name method in localstack

Best Python code snippet using localstack_python

vertex_ai.py

...89    name = "Export Model"90    key = "export_conf"91    format_str = VERTEX_AI_MODEL_EXPORT_LINK92    @staticmethod93    def extract_bucket_name(config):94        """Returns bucket name from output configuration."""95        return config["artifact_destination"]["output_uri_prefix"].rpartition("gs://")[-1]96    @staticmethod97    def persist(98        context: "Context",99        task_instance,100    ):101        task_instance.xcom_push(102            context=context,103            key=VertexAIModelExportLink.key,104            value={105                "project_id": task_instance.project_id,106                "model_id": task_instance.model_id,107                "bucket_name": VertexAIModelExportLink.extract_bucket_name(task_instance.output_config),108            },109        )110class VertexAITrainingLink(BaseGoogleLink):111    """Helper class for constructing Vertex AI Training link"""112    name = "Vertex AI Training"113    key = "training_conf"114    format_str = VERTEX_AI_TRAINING_LINK115    @staticmethod116    def persist(117        context: "Context",118        task_instance,119        training_id: str,120    ):121        task_instance.xcom_push(...

rds_inserter.py

Source: rds_inserter.py

...9                                             password=os.environ['password'],10                                             host=os.environ['endpoint'].split(':')[0],11                                             database='frbhackathon2018tf')12        return connection13    def extract_bucket_name(event):14        return json.loads(event)['bucket_name']15    def extract_file_names(event):16        return json.loads(event)['list_of_files']17    def download_file_from_s3(s3, bucket_name, key):18        s3.Bucket(bucket_name).download_file(key, '/tmp/test.txt')19        return20    def insert_into_rds(s3, conn, bucket_name, file_names):21        cursor = conn.cursor()22        cursor.execute('SET autocommit=1;')23        for file in file_names:24            if 'obfs' in file:25                table_name = 'T_CUST_LOAN_OBFS'26            else:27                table_name = 'T_CUST_LOAN'28            download_file_from_s3(s3, bucket_name, file)29            with open('/tmp/test.txt') as local_file:30                header = local_file.readline()31                for line in local_file:32                    print(QUERY.format(table_name, line.rstrip('\n')))33                    cursor.execute(QUERY.format(table_name, line.rstrip('\n')))34            local_file.close()35        conn.close()36        return37    def main():38        s3 = boto3.resource('s3')39        file_names = extract_file_names(event)40        bucket_name = extract_bucket_name(event)41        conn = create_connection()42        insert_into_rds(s3, conn, bucket_name, file_names)43        output_dict = {}44        output_dict["result"] = "success"45        output_dict["key"] = file_names[0]46        return json.dumps(output_dict)47    return_json = main()...

main.py

Source: main.py

1import os2from datetime import datetime3import boto34def run():5    TODAY = datetime.today().strftime('%Y%m%d')6    LOCAL_FILE_NAME = 'tmp.csv.gz'7    extract_session = create_aws_session()8    download_file(extract_session, LOCAL_FILE_NAME, TODAY)9    load_session = create_aws_session(extract=False)10    upload_file(load_session, LOCAL_FILE_NAME, TODAY)11def create_aws_session(extract=True):12    REGION_NAME = 'us-east-1'13    if extract:14        AWS_ACCESS_KEY_ID = os.environ['EXTRACT_AWS_ACCESS_KEY_ID']15        AWS_ACCESS_KEY_SECRET = os.environ['EXTRACT_AWS_ACCESS_KEY_SECRET']16    else:17        AWS_ACCESS_KEY_ID = os.environ['LOAD_AWS_ACCESS_KEY_ID']18        AWS_ACCESS_KEY_SECRET = os.environ['LOAD_AWS_ACCESS_KEY_SECRET']19    return boto3.session.Session(20        aws_access_key_id=AWS_ACCESS_KEY_ID,21        aws_secret_access_key=AWS_ACCESS_KEY_SECRET,22        region_name=REGION_NAME23    )24def download_file(session, local_file_name, today):25    EXTRACT_BUCKET_NAME = 'sfdv-growp-data'26    s3 = session.resource('s3')27    s3.Object(28        EXTRACT_BUCKET_NAME,29        f'{today}_file_for_gp.csv.gz'30    ).download_file(local_file_name)31def upload_file(session, local_file_name, today):32    LOAD_BUCKET_NAME = 'growprog-schildress-test'33    s3 = session.resource('s3')34    s3.Object(35        LOAD_BUCKET_NAME,36        f'{today}_file_for_gp.csv.gz'37    ).upload_file(local_file_name)38if __name__ == '__main__':...

Blogs

Check out the latest blogs from LambdaTest on this topic:

Best 13 Tools To Test JavaScript Code

Unit and functional testing are the prime ways of verifying the JavaScript code quality. However, a host of tools are available that can also check code before or during its execution in order to test its quality and adherence to coding standards. With each tool having its unique features and advantages contributing to its testing capabilities, you can use the tool that best suits your need for performing JavaScript testing.

Nov’22 Updates: Live With Automation Testing On OTT Streaming Devices, Test On Samsung Galaxy Z Fold4, Galaxy Z Flip4, & More

Hola Testers! Hope you all had a great Thanksgiving weekend! To make this time more memorable, we at LambdaTest have something to offer you as a token of appreciation.

What will come after “agile”?

I think that probably most development teams describe themselves as being “agile” and probably most development teams have standups, and meetings called retrospectives.There is also a lot of discussion about “agile”, much written about “agile”, and there are many presentations about “agile”. A question that is often asked is what comes after “agile”? Many testers work in “agile” teams so this question matters to us.

How To Use WebDriverManager In Selenium

How to increase and maintain team motivation

The best agile teams are built from people who work together as one unit, where each team member has both the technical and the personal skills to allow the team to become self-organized, cross-functional, and self-motivated. These are all big words that I hear in almost every agile project. Still, the criteria to make a fantastic agile team are practically impossible to achieve without one major factor: motivation towards a common goal.

Automation Testing Tutorials

Learn to execute automation testing from scratch with LambdaTest Learning Hub. Right from setting up the prerequisites to run your first automation test, to following best practices and diving deeper into advanced test scenarios. LambdaTest Learning Hubs compile a list of step-by-step guides to help you be proficient with different test automation frameworks i.e. Selenium, Cypress, TestNG etc.