AWS Boto KeyError endpoint_resolver in session.py when using multithreading

When using multithreading in Python with AWS boto to get a session I was randomly hitting this error:

File "/var/runtime/botocore/session.py", line 901, in get_component
del self._deferred[name]
KeyError: 'endpoint_resolver'

It turns out boto is thread safe mostly but the default base session is not thread safe. The original way I was hitting the error was with this:

Here I am depending on base session shared across the threads which is not safe.

session = boto3.Session(profile_name=profile, region_name=region)

This was fixed by creating a new base session just for each thread.

session = boto3.session.Session(profile_name=profile, region_name=region)
local_assume_client = local_assume_client.assume_role(RoleArn=local_role, RoleSessionName="session")

Another example:

remote_session = boto3.session.Session(aws_access_key_id=a, aws_secret_access_key=y, aws_session_token=z)
remote_assume_client = remote_session.client('sts')
remote_assume_role = remote_assume_client.assume_role((RoleArn=local_role, RoleSessionName="session")

Comments

comments powered by Disqus