error while running the example.py on docker, why so? And when I run the same file on local, it runs properly, but to fetch 1000 followers/followings it takes 7 to 8 minutes, what can be done to make the process faster?

This issue has been created since 2021-08-16.

selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page crash
from unknown error: cannot determine loading status
from tab crashed
(Session info: headless chrome=92.0.4515.131)
Any help @Altimis

purnima110895 wrote this answer on 2021-08-16

Please do reply @Altimis, if you have any solution on the above issue.

Altimis wrote this answer on 2021-08-16

Regarding the time of retrieving followers/following, there is a wait parameter that controls waiting time between each scroll. Its default value is 2. If you have a good connection, set it to 0.5 or 1 to minimize the time.
There could be another way to retrieve this data faster, which is scrolling first until the end of the page and then retrieve all data in the element. But it could ban the account that you are using... I'll try this later. Technically speaking, the best way would be to retrieve data from the graph directly, instead of using selenium to retrieve it statically.
However, I've never tried to use this lib in docker... So I have no idea.

purnima110895 wrote this answer on 2021-08-16

Thanks for the response @Altimis , that docker issue has been resolved. Need one more help -
Can we apply multithreading on this functions, like if mutiple users are trying to fetch the data at the same time with the same account, is it expandable?
Please let us know if multithreading can be implemented on this @Altimis .

purnima110895 wrote this answer on 2021-08-17

Any help on the above query @Altimis . Please do reply.

Altimis wrote this answer on 2021-08-17

Thanks for the response @Altimis , that docker issue has been resolved. Need one more help -
Can we apply multithreading on this functions, like if mutiple users are trying to fetch the data at the same time with the same account, is it expandable?
Please let us know if multithreading can be implemented on this @Altimis .

Can you point out how did you solve the docker problem please ?
Regarding your suggestion, As far as I know, it is possible to do multithreading with selenium. So it is possible to retrieve following/followers likewise. Although I havent try it yet.

purnima110895 wrote this answer on 2021-08-18

Thanks for the response @Altimis , that docker issue has been resolved. Need one more help -
Can we apply multithreading on this functions, like if mutiple users are trying to fetch the data at the same time with the same account, is it expandable?
Please let us know if multithreading can be implemented on this @Altimis .

Can you point out how did you solve the docker problem please ?
Regarding your suggestion, As far as I know, it is possible to do multithreading with selenium. So it is possible to retrieve following/followers likewise. Although I havent try it yet.

Thanks for the response @Altimis . Regarding that docker-
Actually we were using microservices and this scweet api and in between we had restapi to call this scweet in the microservice, previously used different containers in docker to run the MS and Scweet api, after alot of efforts, tried resolving running MS and scweet api in same container, it worked out.

wprata17 wrote this answer on 2021-08-25

Thanks for the response @Altimis , that docker issue has been resolved. Need one more help -
Can we apply multithreading on this functions, like if mutiple users are trying to fetch the data at the same time with the same account, is it expandable?
Please let us know if multithreading can be implemented on this @Altimis .

Hi @purnima110895, I have done multithreading with selenium-docker. It is working at the same time with 2 nodes. My problem is to load completely the page on container. It takes much time to load completely. I don´t know why. I´m studying yet how to programm. I don't have a lot of programming experience. I'm still learning, but on Treads I used this python code:
First of all, I´ve imported the packages and functions.
t1 = Thread(target=bot1)
t1.start()

t2 = Thread(target=bot2)
t2.start()

More Details About Repo
Owner Name Altimis
Repo Name Scweet
Full Name Altimis/Scweet
Language Python
Created Date 2020-12-14
Updated Date 2022-09-26
Star Count 518
Watcher Count 12
Fork Count 130
Issue Count 62

YOU MAY BE INTERESTED

Issue Title Created Date Comment Count Updated Date
[webpack-cli] Error: Cannot find module 'babel-merge' 0 2021-05-12 2022-09-14
Possible to disable or customize welcome message? 1 2020-08-03 2022-09-24
`read` prints spurious signal warning 9 2019-07-13 2022-09-15
Please unban @coreybruce 0 2021-05-11 2022-08-13
Ability to change channels? 1 2020-12-27 2022-08-02
anybody figured out some of these settings? 3 2021-01-26 2022-08-01
BOHONG BH25Q64 SPI Flash Datasheet down 1 2021-04-19 2022-04-24
Discovery requires Python>=3.8 1 2021-03-09 2022-06-29
Evaluation error: "Unknown property: solution" 2 2021-02-26 2022-08-17
CI build should fail, if crates are missing `![deny(missing_docs]` 7 2022-04-14 2022-08-27
pytorch1.7.0不支持coremltools,但是SiLU又必须用到pytorch1.7.0? 6 2021-02-02 2022-08-10
Ошибка при загрузке данных при проверка на соответствие uuid 0 2022-09-09 2022-09-26
能否添加一个面板设置ssh_key的位置? 0 2022-03-30 2022-07-08
安装过程 添加管理账号 密码无法输入 6 2022-02-21 2022-09-15
Sniper increase default gas limit 0 2021-07-18 2022-09-09
GetPortfolio в PortfolioResponse.PortfolioPosition[].figi для валют заполняет не figi, а ticker 4 2022-01-18 2022-07-08
Button icons not showing 1 2022-02-01 2022-09-14
Blocked account keeps trying accessing the original image and gets error "Wrong MIME" 1 2020-03-10 2022-08-12
Home page counter bug 12 2022-06-01 2022-09-17
Prefer low-latency peers 0 2021-01-23 2022-09-21
Bezels not working & Buttons have strange white boxes 13 2021-09-02 2022-09-12
Background not generating 1 2021-08-06 2022-09-12
Problem when installing FatRat 4 2020-09-07 2022-09-26
fatrat command not found 3 2020-11-25 2022-09-18
[Bug?]: Cannot generate enum types with GraphQL Codegen 3 2022-07-23 2022-09-18
[Bug]: В GridAvatar не прогружаются аватарки 0 2022-09-07 2022-09-17
ckan 2.8 compatibility 1 2018-10-10 2022-09-23
Show embedded cover art [iOS] 1 2022-01-06 2022-06-16
What is colour in biomes.json? 1 2021-08-12 2022-09-20
ALB: AssociateAclsWithListener/DissociateAclsFromListener InvalidAclIds error 1 2022-04-11 2022-09-24
Integrate docs into existing OZ docsite 0 2022-04-06 2022-09-21
Separate local and offline sections; they have distinct needs 0 2021-12-13 2022-09-18
Feature Suggestion: User Snippets 0 2021-09-30 2022-09-20
Viewport Transparency 3 2021-08-07 2022-09-13
Icon Request: Fediverse 0 2022-06-07 2022-09-18
Support AS923 in device provisioning CLI 1 2022-01-06 2022-09-25
[BUG] Bokeh 3.0.0 cannot serialize Pandas Dataframe 3 2022-07-13 2022-08-22
[IT DateTimeV2] Mentions in the form "day-of-week, date" not recognized 1 2022-03-12 2022-09-13
[TR Currency] Term "abd dolar" not recognized 0 2022-03-12 2022-09-08
Correction of widget's method in documentation. 0 2022-03-08 2022-09-13
[Bug Report] ArrayItems Validation not implemented 0 2022-09-22 2022-09-21
Traits and spurious errors 1 2022-09-25 2022-09-27
regression: TLS linker issues 4 2022-04-16 2022-08-06
ECONNRESET at TLSWrap.onStreamRead issue impacting Azure extensions zip deploy (related to Node v16?) 5 2022-04-01 2022-09-28
Port last year's stop event lagged calculations from R to PostgreSQL 1 2019-06-30 2022-08-27
Could not run command: cmpress 5 2018-06-14 2022-08-03
runtime error: invalid memory address or nil pointer dereference 2 2021-12-20 2022-09-20
Why is so important to contact the local node? How do you ensure startup order? 5 2021-05-24 2022-09-23
docker image from `sbt docker:publishLocal` fails with "Failed to create parent directories for [/opt/docker/./logs/application.json]" 6 2020-08-19 2022-08-27
Full AMQP url inside generated secret 1 2022-07-07 2022-09-17