Telescope: Elasticsearch and Automatic Deployment

For the 0.7 release, I've been working on a couple of issues: adding text indexing and automatic deployment.


Text indexing
Adding text indexing to Telescope was interesting. After checking out some options, Elasticsearch (a distributed, RESTful search and analytics engine) was the tool that we decided to use to implement text indexing. I added a new container running Elasticsearch to our workflow that will be waiting for requests to index blogposts or search for already indexed blogposts that contain a passed string.

Surprisingly, the main problem I had to deal with was not functionality or interactivity with Elasticsearch, but mocking it. Elasticsearch works great with Telescope, but a fourth of our tests fail due connectivity problems (Elasticsearch is down when I run tests, so the client complains). I did a bit of research to find out about use an Elasticsearch client in mock mode to mimic what we do with redis, but from what I've seen it's not as easy as I thought it'd be and it might involve adding some Java to the mix, which is something that we don't really want for Telescope.

I'm still working on a solution (I'll have a PR ready soon), but I think I'll wrap the Elasticsearch pieces in our code in conditionals that will only run when an environment variable is set to a specific value.


Automatic deployment
For 0.7, we agreed to add simple auto deployment. The ultimate goal is to have blue/green deployment, but since that'll probably involve deep changes to our current docker settings I think it's a good idea to take this first step and then try to build on it in future releases.

To implement automatic deployment, I added a small server that waits for requests from GitHub to shutdown, reclone, and redeploy Telescope whenever a pull request is merged to master. For this, we're using GitHub webhooks (Calvin I worked on this part together).

I got this piece to work on my machine, but I still have a small issue with our .env file. Hopefully I'll be able to fix it soon and add this to our staging server soon.


Other issues
I also did a bit of work to have persistent storage with redis and adding some more filters and nested queries to our GraphQL. I think these 2 will very likely be included in the 0.8 release.

Read More

Telescope: Up and Running

Yes, Telescope is ready to start taking its first steps!

After a couple of weeks of hard work, Telescope has finally become a functional piece of software. All the elements, or I should say all the effort made by all the contributors, was finally put together and now it is possible to run telescope and see it do what it was created for.

This is really awesome, seriously. Running Telescope and being able to read posts on it is amazing. Here, take a look:

This has been so far a really awesome experience. The learning process, the collaboration, the enthusiasm from all the people involved... I'm very happy that this project exists, and I'm even happier that I'll be able to keep working on it.

As I said, the last couple of weeks were crucial to be able to get to where Telescope is right now. The components responsible for fetching feeds, process and store them in our database were connected to the front end through our web server, using 2 endpoints that I added. I had a lot of fun adding those 2 endpoints and tests for them. I had to learn about how to implement paging and adding link headers to the server's response, and a little bit of redis, which were both completely new to me, and added valuable knowledge to be used in the future. I also had the chance to get involved with the docker side of the project. 2 awesome contributors worked hard to add redis and SAML2 support to our docker workflow, and I was fortunate enough to have the chance to add some suggestions and comments here and there.

I think I have a better understanding of how redis and SAML2 can be incorporated to a project, and although I've worked with docker before, I feel more confident about using it now. That's also another part of this project that makes it so valuable. I started contributing to Telescope at the very beginning, and just by reviewing other people's work, I had the chance to learn so many new things. As an example, before telescope, my relationship with CI was almost non existent, but just by reading other people's code, their comments, and following all the steps they had to go through to add CI to Telescope, I'm ready to start adding my own contributions in this area. That right there is invaluable experience for my future in this industry.

So there it is, Telescope is really getting there. Yes, bugs will come up, adjustments will have to be made, but it is ready to see it work and do what it's meant to do. Isn't it exciting?

Good hacking and have fun!

Read More

Telescope: growing faster than expected

Telescope has been alive for 2 weeks, and its development couldn't get more exciting!

I mentioned in a previous post how the project seemed to be a bit chaotic at the beginning, mostly because of the number of people involved in it and the different sets of skills and experience that each contributor has.

Well, that was last week. This week, we have a very enthusiastic group of developers exchanging ideas, offering help, being passionate about their contributions, and all that with a surprising level of organization completely unexpected (at least to me) for a group this big, and we're taking about between 50 and 60 people. As I said before, it's been fascinating to watch it grow, and I couldn't be happier to be part of it.

But there's more. At a personal level, I've been able to contribute to many parts of the project, and in may different ways! Back-end, containers, documentation, testing, CI... all that in just 2 weeks! In some cases, I added features that will enhance the capabilities of the project, in others I just made small changes or suggested someone a different approach for what they were working on. It's been a very enriching experience because not only I had to review and refresh previous knowledge to be able to offer some help, but also I had to do a significant amount of self-teaching so I could take on certain issues and complete some tasks.

So far:

• I added a tool to fetch data from Github, usign GitHub's API
• Implemented tests for that tool, using mock responses
• I tweaked our .eslint a little bit
• I fixed some linting errors
• And I tried to add husky to the project, but another contributor did a way better job, so it was better for the project to prioritize his pull request over mine

Also, I reviewed other people's contributions, I collaborated with other developers to add docker to the project and write documentation about how to use it, I gave some advice about how to add tests using mock responses... and this is only the beginning! In the next 2 weeks, I'll be able to do some more work, and the best part is that even if I'm not directly involved in some part of the project, I'll be able to review or at least take a look at what others did to solve some problem and learn a lot from it. What else could I ask for?

The only downside of all this is that I got a bit too distracted and I didn't contribute to another projects, as I was hoping to. My plan was to work on a PenguinV's issue, but since I didn't, I'll try to get it done this week, and hopefully take on another PenguinV's issue next week.

As usual, good hacking and have fun!

Read More

Telescope: A tool for tracking blogs

Have you ever heard of PlanetCDOT?

Well, PlanetCDOT "is a collection of current blog posts by students, professors, and researchers at the Seneca Centre for Development of Open Technologies and related organizations around the world", and it has been around for a long time. The technology behind PlanetCDOT is Planet, "a feed aggregator application designed to collect posts from the weblogs of members of an internet community and display them on a single page", and according to the planet index page, it hasn't been updated in 13 years.

The good thing about software that hasn't been updated in more than a decade is that there's plenty if not all of the room for improvement, and now there's a project going on to create a new aggregator for PlanetCDOT.

This project is Telescope, and It's hard to describe what it is right now since what we have is a varied group of developers, with different set of skills, trying to put together a fairly complex piece of software. It started as a disjointed mess, but it's been fascinating to watch how the people involved in it started to file issues, submitting pull requests, reviewing other people's work, discussing approaches, all in a very organic and natural way,

I'm one of the lucky ones working on this project, and so far it's one of the most interesting collaborative experiences I've ever had. I must confess that I spend hours just watching the project grow, keeping track of how others review code, suggest solutions, help each other... and with watching I mean literally staring at the issue and pull request trackers to see the new additions. As I said, it's truly fascinating.

At the early stages of the project I thought I wanted to take care of .env files, using dotenv, but someone was faster than me and already solved that problem. My current intention (hopefully I'll be able to take care of this before someone else does) is to create a module that will fetch information related to a specific github repository/pull request/issue using GitHub's API. I have some experience with it, and since someone suggested that it could be a good addition to the project I thought it was a good fit for me.

I'm aware my description of Telescope is very vague, but I'd like to wait until a few more pieces get merged to the project, adding more functionality, and then I'll give more information about the features we'd like Telescope to have, and how the contributors are coming up with creative and interesting solutions.

Being part of Telescope is great, but I'd also like to work with other projects, and since my collaboration to PenguinV was welcomed, I think I'm going to try to take on another issue of theirs. This one is going to be tough!

As usual, good hacking and have fun!

Read More

Hacktoberfest: Documentation

We're in the last week of Hacktoberfest!

So far, participating in Hacktoberfest has been great. I've been able to contribute to very different projects and work towards the goals I had set for myself.

My work on Dashboard allowed me to improve my web skills (not as much as I'd have liked, but still) and collaborate in a project that has been around for some time. It's not a big project, but it's a very interesting one because of all the different components that it has and how useful it is for the people involved in it.

PenguinV was my chance to collaborate in a project that I really like, and also a great opportunity to face a real, scary challenge. Having to contribute using OpenCL, something I wasn't familiar with at all before, was one of those things than can easily make you think that you shouldn't be in this field.

For my last contribution, I wanted to work on PenguinV again, but since we're at the end of the month and the PenguinV community hasn't responded (yet) when I told them that I wanted to work on some issues, I decided to look around to see if I could find some other interesting projects.

And I found ImCompressor. ImCompressor is an image processor that works with lossy and lossless compression. What I found interesting about this project is the fact that it's a very young one. A small group of developers started it a month ago. All my other contributions during Hacktoberfest were related to bigger projects that have been around for a while, so I thought it could be interesting to contribute to a young project and maybe become a core part of it. 

When I found the project, they were getting ready to release version 0.8, so since there wasn't much that I could offer (remember, it's a very young project), I mentioned to the team that my first language is Spanish and that I could add a Spanish translation. They welcomed the idea, and I thought it was a good way to introduce myself, getting familiar with the project, and keep an eye on new issues popping up so I could work on them and become a main contributor.

Oh, before I forget, when I started the translation I realized the ImCompressor team uses Poedit, a cross-platform, open source, translation editor. Another interesting project, I also want to keep an eye on it and maybe see if I can contribute to it in the future.

I like ImCompressor. I've had issues with lossless JPEGS in the past, and I think this project could help many people. I hope it won't turn out to be one of those many projects that dies a few months after being born.

Well, and that's all for this month. As I said in another post, I really want Hacktoberfest to become a yearly tradition, but not the only excuse to keep collaborating with open source projects.

As usual, good hacking and have fun!

Read More

Hacktoberfest: WIP

Hacktoberfest, week 3.

Only a few days left, and I just submitted my third PR.

I was considering giving home-assistant another go, as I mentioned in my previous post, but there are so many issues out there that I thought that maybe it wasn't worth it.

For my first issue, I enhanced the capabilities of CDOT's Dashboard, and I decided to take on the next related issue for that part of the project: update the front-end.

I updated the front-end with a new panel that shows pull requests that are pending for review. Hopefully this new panel will help the team keep track of pull requests and it'll speed up the workflow.

One of my goals for this Hacktoberfest was to improve my web development skills, and this issue was going to be my opportunity to do that. Unfortunately, it didn't turn out the way I was expecting. I spent very little time writing React and most of the time doing CSS. Yes, it gave me the the chance to learn some new CSS tricks, but it also helped me remember why I've been putting off improving my web development skills and why I've always thought that CSS is a creation of the devil.

Dashboard with the new panel

My pull request has been accepted and merged, so I'm expecting to see the new panel up and running any time soon (to be honest, that's very exciting!).

So far, Hacktoberfest has been a really enjoyable and amazing learning experience, and I'm almost sad that it's going to be over in a week. I really want it to become a yearly tradition, if time permits.

As usual, good hacking and have fun!

Read More

Hacktoberfest: good first issue

It's Hacktoberfest time!

As I mentioned in my previous post, I'm very excited about Hacktoberfest, not only because I'll be able to contribute to some cool open source project, but also because I'll have the chance to improve  and test my skills, and maybe try to take on some scary project that will force me to learn and get familiar with new technologies.

My intention was to work on some issues that I found interesting and relevant for the type of skills that I'd like to develop, and also getting familiar with some interesting projects.

Unfortunately, things rarely go the way you want in software development, so I had to come up with a plan B for my very first Hacktoberfest issue.

At first, I wanted to work on an issue in the home-assistant project, but I had some problems setting up the development environment, and since they haven't replied yet, I thought I could work on something else while I wait for their advice.

Since I was running out of time, and all the other issues that I wanted to work on require more than a couple of days of coding, I turned to a project that I've worked on before, knowing that I could find some interesting issues that I could fix within the available time that I had.

This project is Dashboard, maintained by CDOT's RAs.

Dashboard project (running in debug mode with fake data)

Dashboard displays information about the projects that are currently in development at CDOT, along with some infrastructure information and scheduled events. It populates some of the panels with data fetched from Github using Github's API.
For the PR Reminder panel (WIP) and the Help Wanted panel (coming soon), it was necessary to filter some of the data fetched from Github's API so the only issues and pull requests used to populate those 2 panels would be the ones with the appropriate labels. The issue that I worked on specifically requested filtering out issues that didn't have the Help Wanted label and also making sure that all the used issues included a priority label so they could be appropriately sorted.

It was a quick fix, but a very important one since the Help Wanted panel will be up soon (that's another issue I'd like to work on for Hacktoberfest), and all the RAs at CDOT will be able to have updated and sorted information about issues that might need help (hence the Help Wanted label) and/or pull requests that need review.

Hopefully, I'll be able to work on home-assistant's issue next week, but if I can't, I might take on a penguinV's issue that will help me get familiar with their code base and maybe I'll be able to contribute more in the future.

So, good hacking and have fun!

Read More

Open Source collaboration at its best

Remember mininote? It’s that small app that we made last week. It’s also part of an assignment for an Open Source course that I’m taking.
So, what about miniNote? Well, since we’re here to talk about Open Source, miniNote is the perfect excuse to show how an Open Source project grows thanks to a community of people who want to help improve software and make the world a better place.

As part of the Open Source course, we all (my classmates and I) had to create a miniNote, and the best part is that now we also got to check each other’s projects and fix bugs or add features to them.

I worked on 3 different projects:

Grommers00/my-note

my-note had a very minor issue that I wanted to address. The script tag was located at the beginning of the body tag in the html document, and I thought it was a good idea to move it to the end of the body.

It’s a minor bug, but depending on the size of the html document, a script tag in the wrong place can give the perception of a slow, bloated website.


jerryshueh/simple-note

For simple-note, I thought adding a save-on-disk functionality would be a good addition. Simple-note already had a saving functionality (not in a file) that uses ‘ctrl+x’ to save, so I added ‘ctrl+s’ as the shortcut to save on disk. It was an interesting addition since it was my first time using ‘FileSaver.js’.


Cindyledev/my-note

For this assignment, I was supposed to collaborate with 2 projects, but after finishing my 2 contributions, I started looking around because I was curious about other people’s contributions and how our apps were getting better thanks to mere collaboration. Checking my-note’s issues, I saw that one of the issues was filed by the creator of the project, not by a classmate trying to fix a bug or add a feature. In this case, it was a “real“ issue that was preventing the creator of the project from finishing the app and continue with the assignment. A fellow programmer stuck with a piece of code in an Open Source project. Say no more.

The issue was that the saving functionality wasn’t working, and the app wouldn’t show saved notes after refreshing. I did some testing and switching ‘querySelector()’ to ‘getElementById()’ seemed to fix the problem.

One mistake I made in this one is that I added the meta charset to the html document as part of this issue instead of opening a new one. I don’t think it’s a big deal (in this case), but I should be careful in the future because a contribution done through the wrong issue is almost untraceable, and if it creates a bug, it will be a huge headache for whoever has to deal with it.


I think we could call this part of the assignment pure Open Source collaboration at its best. A group of people helping each other improve their software by fixing bugs and adding features. Isn’t it amazing and exciting? No? Well, I think it is.

Read More