Console #174 - Interview with Alicia of Web Check - OSINT tool for analysing any website
Featuring Watlings, Tidal, and novel writer
🤝 Sponsor
This space is reserved for sponsors that support us to keep the newsletter going! Want to support Console? Send us a note at osh@codesee.io
🏗️ Projects
Browse through open source projects on OpenSourceHub.io, add your project to get more exposure and connect with other maintainers and contributors!
Watlings
Learn WebAssembly by writing small programs!
language: JS stars: 1168 last commit: 2 days
repo: github.com/EmNudge/watlings
Tidal
Tidal is software for making patterns with code, whether live coding music at algoraves or composing in the studio.
language: C++ stars: 1952 last commit: 3 weeks
repo: github.com/tidalcycles/Tidal
site: tidalcycles.org
novelWriter
novelWriter is an open source plain text editor designed for writing novels. It supports a minimal markdown-like syntax for formatting text. It is written with Python 3 and Qt 5
language: Python stars: 1542 last commit: a week
repo: github.com/vkbo/novelWriter
site: novelwriter.io
Web Check
All-in-one OSINT tool for analysing any website.
language: TypeScript stars: 5591 last commit: today
repo: github.com/Lissy93/web-check
site: web-check.xyz
Join thousands of other open-source enthusiasts and developers in the Open Source Hub Discord server to continue the discussion on the projects in this week's email!
🎙️ Interview With Alicia of Web Check - All-in-one OSINT tool for analysing any website
Hey Alicia! Thanks for joining us! Let us start with your background!
Heya 👋 I’m Alicia Sykes (Lissy93 on GitHub), a full-stack developer from London UK. I got into coding quite early on. From the first time I used a computer (at about 11), I was fascinated by how they worked. School was never my strong point, but something about computers just clicked for me, and I've been hooked ever since.
I think it's super cool, how you can use code to create things which solve real-world physical problems, without needing anything more than just a laptop (and maybe some caffeine!).
This was back in 2005 - the days of dial-up and before computing was taught in schools, so to learn to program I'd get these massive 500-page books from the library, and read them through line-by-line! Course, it's much easier now, but things also move a lot quicker, so I'm still learning everyday.
In terms of languages and frameworks, it's been really varied over the years - I started out with ActionScript and Java, before moving more into web technologies (with, don't laugh, PHP!). My first real job was as a mobile developer. Even still my tech stack is quite mixed, as it depends on what I'm building - I love Svelte for web, Flutter for mobile, some Rust and Go for backend, join it all together with Docker and a bit of good ol' Bash script.
Who or what are your biggest influences as a developer?
I always continue to be inspired by all the open source developers, focusing their time and effort to build the foundations. Seriously, none of what we do would be possible without the frameworks, libraries and tools we use on a daily basis, yet the developers behind them get very little, or sometimes no recognition.
What’s your most controversial programming opinion?
If you're not having fun, you're doing it wrong!
Programming isn't just about slinging code and churning out apps. It's a creative process, a way of solving puzzles, and a means of expressing yourself. When you're enjoying what you're doing, you're more likely to think outside the box, find innovative solutions, and, let's be honest, write better code.
Now, I'm not saying it's going to be a laugh a minute, but I have noticed that devs who genuinely love what they do tend to produce better code than those who just clock in and out of a 9-5. So if you find yourself not enjoying your current project, maybe it's time to shake things up a bit—approach it from a new angle, experiment with a different tech stack, or pivot to something else entirely
What is your favorite software tool?
Ooh, that's a big one! I actually like to keep things pretty simple, my every-day setup is Linux (Arch, btw!), Firefox, Neovim and VS Code and the terminal.
My favorite tools right now, are probably z, fzf and thef*ck. I love CLI utilities that make life quicker and easier (if you do too I recently made a list of several others https://dev.to/lissy93/cli-tools-you-cant-live-without-57f6 )
Why was Web-Check started?
When you look below the surface of a website, there's a plethora of interesting information exposed that shows how it's been built, what architecture it's running on and how it's connected to other services.
This data can be super valuable to developers, sysadmins, security researchers, OSINT investigators, marketeers, site owners, or anybody who's just a bit curious.
I created Web-Check, as there wasn't yet any single tool which is able to fetch, analyze and collate all this information in one place, and without needing any special setup or technical skill from the user.
How does Web-Check work?
At a high-level, it's a series of lambda functions, which are each responsible for a single task. The user enters a URL which they'd like to scan, requests to the serverless functions are fired off asynchronously, and the UI is updated as and when new data comes in.
Who, or what was the biggest inspiration for Web-Check?
I love tools which collate present data in interesting ways, and let you, the user uncover trends that wouldn't otherwise be clear. I have to give a shoutout to Shodan, Archive.org and Cloudflare Radar in particular. If this kind of stuff interests you, I've made a list of more of them here: https://notes.aliciasykes.com/18611/fun-with-real-time-data
Are there any overarching goals of Web-Check that drive design or implementation?
Yes,
Easy to use - Users should just be able to paste a URL in, and everything should happen automatically, with results made easy to understand for a user of any level
Open and transparent - I want it to be clear to users how it's working, how the data is fetched, and what the results means
Free and open source - The service is free to use, and anybody can take the code, modify it and deploy it themselves
Are there any competitors or projects similar to Web-Check? If so, what were they lacking that made you consider building something new?
There’s nothing which does everything all in one-place, or that’s fully free and open-source. But there are plenty of other useful tools which let you learn more about a website or domain. I’ve linked to them within Web-Check (https://web-check.xyz/about#additional-resources), I think my favorite of those are Archive.org, Cloudflare Radar, domaintools.com and Shoan.
What was the most surprising thing you learned while working on Web-Check?
I think just how much info can be gleaned from analyzing the response of a single HTTP request to a server. You can understand where and how a site is hosted, what it’s built using, which security configurations have been applied, what the performance is like, who's behind the site, and so much more.
What is your typical approach to debugging issues filed in the Web-Check repo?
I try to go through issues most mornings before work, triage everything and ask for more info when needed. Then in the evenings when I have more time I go through and submit PRs to fix any bugs raised.
If I can reproduce an error, it's usually quite straightforward to fix. That's why it always helps when users explain exactly what isn't working, steps to reproduce, actual output + expected output, and details of their environment. Error messages or screenshots are a bonus!
Is Web-Check intended to eventually be monetized? If so, how?
Right now, Web-Check is not monetized at all. Actually, none of my projects are. Instead, I receive a small amount of funding from GitHub Sponsors, which is just about enough to cover server costs, and I am super grateful to my sponsors as it's them who keep apps like this free for everyone else.
Usually my motive behind the projects I build is: 1) To solve a problem, 2) to help people better understand privacy & security, and 3) to have fun! Of course, this isn't always the most sustainable, and sometimes I do consider monetization.
If I were to go down the monetization route with Web-Check (or any of my other projects), I would still want all of the code to remain 100% free and open source, and I would never implement anything (like certain ads) which compromise the privacy of my end users. That somewhat limits the options slightly, but one approach would be to have a free usage tier on the managed instance, and then charge for additional scans, while still allowing more tech savvy users to self-host their own instance if they wish.
What are you most proud of?
In the first 2 weeks after launching Web-Check, we had 100,000 unique visitors, which I was very happy with (and quite surprised!). This momentum has kept up, and although the project is only a few weeks old, we're at 5k stars on GitHub, with a solid and consistent number of returning users.
How do you balance your work on open-source with your day job and other responsibilities?
That's a good question, and it's something I've still not properly fully figured!
I usually get up quite early, have a nice cuppa tea, and work my way through issues and PRs before heading to work. Then in the evening, that's when the real coding happens! I probably spend about 4-6 hours most evenings working on various open source projects (including Web-Check, Dashy and some Linux stuff). I think the key is to have fun, if you enjoy something (like open source), you'll naturally find a way to fit it in, like you do with video games, the pub, and everything else you enjoy.
What is the best way for a new developer to contribute to Web-Check?
I would very much welcome any first-time contributors to check out Web-Check! The best way to get involved is simply by submitting a PR to either fix a bug, add a feature, or improve upon any part of the code. There’s build and deploy instructions in the repo, but if anyone needs a hand with their first PR, I’m very happy to help!
Where do you see software development heading next?
AI is going to be a big part of future software development. I don't believe it's going to take our jobs or anything like that (at least not yet!), but it's going to be more important than ever to keep up with the changing scene, and to leverage these tools in order to not get left behind.
And, on another topic, there’s the question of diversity. The tech world has been a bit of a boy's club for too long, if you ask me. I'd love to see open source leading the way in creating a more inclusive and diverse community, where everyone has a chance to contribute and have their voice heard.
Do you have any suggestions for someone trying to make their first contribution to an open-source project?
- Choose a project that excites you - it's a lot easier to contribute to projects that you are genuinely interested in
- Start small, either fix a typo, add to the docs, implement an issue (look for tickets marked as good-first-issue)
- Communication is key - talk to the maintainers and community, listen to advice and make friends
- Don't be disheartened by setbacks. It's frustrating when a PR is rejected, or someone beats you to it. But the important thing is to keep at it, learn from your experiences, and enjoy the ride.
What is one question you would like to ask another open-source developer that I didn’t ask you?
What's the most unexpected thing that's happened to you because of your involvement in open source
What threats to open source coding do you see on the horizon
What threats to open source coding do you see on the horizon
I think that Microsoft’s takeover of GitHub really highlighted the value of open source. But not in the way that we might hope. They acquired GitHub for almost $7.5 billion, that’s over 25x its revenue - why? Because the content hosted there (both code and related usage data) is extremely valuable to them. We saw how they turned open source contributions into a commodified product with the introduction of GitHub Co-Pilot (an AI trained on GitHub data), but I believe that’s just the start.
Another key threat is that of developers becoming frustrated by corporations who take their work, and re-sell/ distribute it as their own, often making hefty sums, while the maintainers who are doing all the work receive nothing (except maybe a few feature requests!) - not even attribution. Recently I’ve seen a sharp increase in community projects being deleted / taken private for this very reason (like Wappalyzer).
Want to join the conversation about one of the projects featured this week? Drop a comment, or see what others are saying!
Interested in sponsoring the newsletter or know of any cool projects or interesting developers you want us to interview? Reach out at osh@codesee.io or mention us @ConsoleWeekly!