Did you learn PYAUTOGUI with Colaboratory isn’t working? Chances are you found a bug using pyautogui in colaboratory and because you’re using pyautogui in colaboratory, you will get a bug. I wrote this blog for you.
PyAutoGUI is a cross-platform GUI automation Python module for human beings. Used to programmatically control the mouse & keyboard.
source – https://pypi.org/project/PyAutoGUI/
Notice mouse and keyboard are a part of the control.
Unless you’re an employee at Google and plug your mouse or keyboard into this server, there’s a really high chance you’re not going to make this work…
Here’s your environment right now; you’re using a computer that has a browser that connects to googles jupyter notebook solution they call colaboratory, pyautogui is aiming to use your mouse and keyboard on the local machine. If you’re trying to use pyautogui on googles colaboratory, you’ll be able to install pyautogui no problem (yikes), import into the code (even thought you can’t logically use the code), and when you try to use the minimum requirements of the language you’ll fail; mouse & keyboard.
The ideal; situation is you have pyautogui running on a local machine that has access to these mice/keyboard, monitor, and then you’ll be able to use pyautogui. I like running my pyautogui in pycharm or jupter notebooks.
The wireframe below is built to illustrate the typical environment for someone accessing this solution.
My suggestion is you put jupyter on your computer instead using anaconda distribution, which packages a lot more into the implementation and will help you solve more problems faster. PLUS give you access to a massive amount of cool tools.
I’m going to suggest the next path of learning will be install anaconda distribution, which leads to jupyter notebooks being installed on your computer because this direction you’re on with colaboratory is regrettably dead end. The goal here is to show you that by installing anaconda distribution you get jupyter notebook and it’s maxed out for your usage. Also, pyautogui will work. Below I explain in more detail this drawing.
Installing anaconda distribution gives you access to a lot of tools and you will be able to use pyautogui with the same interface as googles colaboratory product.
My Google colaboratory can’t work because it doesn’t have an active display?
An active display like the monitor you’re reading this content on right now.
Without an active display you’re unable to see your mouse and your typing would be pointless right? You’d not be able to see anything happening. That’s similar to what’s going on here, if you want to think of it that way. BTW, the diagram above explains the mouse/keyboard failures.
This is why pyautogui can not work with Google colaboratory. Your monitor isn’t technically plugged into the google server. You’re rending html, css, javascript and getting website in return. Your web browser renders my blog as something fancy and not a chunk of code.
When you load colaboratory, imagine you’re loading things on their product, in the cloud, which your active monitor is not plugged into. Rather your web browser is plugged into colaboratory, a website, and if your minimum requirement is to complete this in google colaboratory then I understand why you would not want to install the anaconda distribution.
Anaconda distribution is what Colabratory is built on.
I’ve done the research and found out this information for you. Yes, jupyter notebooks is being used in colabratory. If you’re trying to use pyautogui, you’re going to need an active display, install it local and begin.
I also tried to make pyautogui work in Googles app because I too wanted to do macros, build bots, and save time automating boring stuff.
Colabratory by Google is built around and using the anaconda distribution’s solution, jupyter notebooks… and it’s intuitive to know about installing the anaconda distribution because it helps you jump into this solution without these errors generated using a cloud machine that you don’t own on the internet.
This screenshot below shows how I start jupyter notebook local, it’s easy to use and pyautogui works great on mac and both windows. The links I’m providing are links to blogs I wrote and it’s a walkthrough to help people walkthrough the anaconda distribution installation.
A screenshot of jupyter notebook starting. An alternative to using a workspace that’s similar to colaboratory.
FYI; Setting up anaconda distribution will allow you to use pyautogui better than colaboratory and on your local computer. Your computer local has a display which will help you avoid these errors.
Have you ever been in a forest and thought it would be okay to try something new and because no one is around it makes it easier to attempt trying something new?
Something about being in the woods is freeing and we can finally be a little cave man.
No one is filming. No cameras watching. No one to impress.
Similar to writing python on your local computer or in a code editor, ide, cloud based notebook, or whatever you choose. It’s not exactly big data sql.
Often times in the forest we are left with many options and none of them are the most optimized for success. Thus we are here with this blog.
Using Pyautogui with Colaboratory is not currently possible. The code isn’t running with an active display.
It’s okay we did this because now we know better and learned our lesson. If you think this is a little funny that we both fell on the exact same problem, please add me on twitter before you leave.
Pyautogui is going to look for an active display to do the work you’re asking and colaboratory is not built to do your task.
Colaboratory is a web based place to run python and collaborate with others via a quick share link…
At this point, you’re probably in a project or having to sell this to someone who told you to accomplish this task.
Wiki; Google Colaboratory (also known as Colab) is a free Jupyter notebook environment that runs in the cloud and stores its notebooks on Google Drive. Colab was originally an internal Google project; an attempt[29] was made to open source all the code and work more directly upstream, leading to the development of the “Open in Colab” Google Chrome extension,[30] but this eventually ended, and Colab development continued internally. As of October 2019, the Colaboratory UI only allows for the creation of notebooks with Python 2 and Python 3 kernels; however, an existing notebook whose kernelspec is IR or Swift will also work, since both R and Swift are installed in the container. Julia language can also work on Colab (with e.g. Python and GPUs; Google’s tensor processing units also work with Julia on Colab[31]).[32][33] (Updated with wiki definition as of jan3,2022, you know these things change if anyone wants to change the page.) It’s likely people like the colabratory environment because it seems like a great solution for gui automation, however if you want to do local based macros you may want to considering going down the same route of Google during the creation of this product, explained above in the wiki definition (you didn’t skip). Explaining code is a good thing, wanting to run things this way may be better in anaconda distribution, which is the same route as google.
Code explanation is a good thing, and explaining code – that’s not a bad thing when trying to explain crazu gui automation, I know because I use it to help me document local based gui automation. If you want something with a similar feel, please install anaconda distribution on your local computer and then setup your first notebook. Anaconda distribution has a lot of applications (big focus on data science engineering, visualization, and productivity applications for data gurus) and if you only want one application (notebooks), consider this path before installing anaconda distribution on your computer.
Install Alert? Get IT help if you’re at a company that has an IT department..
It’s rare an IT team doesn’t know about someone using anaconda distribution internally, externally, with fancy widgets or they use it themself.
Best to consider how they do this at your business (and if they don’t, ask for permission), they being your IT team, information tech team (maybe they call them engineering),… Please do the contacting of the appropriate people before trying to develop the app you want to build in this app I’m suggesting you install.
What you’re building may not be allowed in jupyter notebook on your machine right now, and maybe an engineer should get involved?
Follow along on my Colaboratory.
Being new at python, is where everyone will be at some point, and just discovering Colaboratory it’s easy to test a library that will pip into the environment.
The Jupyter Notebook has become a popular user interface for cloud computing, and major cloud providers have adopted the Jupyter Notebook or derivative tools as a frontend interface for cloud users.
Examples include Amazon’s SageMaker Notebooks[26] Google’s Colaboratory[27] and Microsoft’s Azure Notebook.[28]
Everything below is for people who need a little more elaboration, a screenshot, or more information to feed to someone who may be asking you questions about why you’re moving to a different solution.
Colaboratory is a nice place to add code related to pyautogui, and explain pyautogui, a lot like anaconda distribution, but it will not be a place for GUI automation when cloud/server based backend is running the python as it does not have a display, being that a server is running the code, and this server is ‘google’ and not your computer, there’s a hiccup here… you don’t have a relevant display… Literally none what-so-ever.. Just look at that bug, just freaking look at it.
PyAutoGUI in Colaboratory looking to show what’s happening when you try pyautogui and there’s no DISPLAY, something it’s expressing in line 160 sucks.
What is colaboratory?
Colaboratory is an online colaboration tool that lets you write your python code, add files, document well, and did I mention document python.
Follow along with me here for any additional support.
DM me on Twitter.com/itylergarrett for more support! Happy to share your problem with my following on twitter to help you get quick visibility into any problems you’re facing, we are a team when it comes to problem solving, and chances are there’s an easier path.
Anaconda distribution might be overkill and you may only need jupyter notebooks. However understanding anaconda distribution is great path for anyone using apps like pyautogui on colaboratory in my humble opinion (12 years enterprise data tech consulting and project/product management). Next you’ll likely be excited to start using the community edition on pycharm for when you are ready to deploy these scripts to run on your active monitor. Next will be how to automate pycharm scripts.
Best of luck automating key strokes and various things in pyautogui. I recommend keyboard for a nice alternative to pyautogui!