Trending tech aggregator http://nextbigthing.tech
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Steven vanZyl f1bb90e677 Switch to Ruby and Literate Programming 1 hour ago
views Switch to Ruby and Literate Programming 1 hour ago
.gitignore Switch to Ruby and Literate Programming 1 hour ago
Gemfile Switch to Ruby and Literate Programming 1 hour ago
Gemfile.lock Switch to Ruby and Literate Programming 1 hour ago
LICENSE Add license 5 days ago
README.org Switch to Ruby and Literate Programming 1 hour ago
Rakefile Switch to Ruby and Literate Programming 1 hour ago
main.rb Switch to Ruby and Literate Programming 1 hour ago
notes.org Switch to Ruby and Literate Programming 1 hour ago

README.org

Intro

Agenda

DONE Look for Competition

CLOSED: [2019-11-15 Fri 13:12]

If it already exists then it might not be worth doing. Or maybe it will be, who knows?

I don't think there is any site quite like what I am going for.

DONE Evaluate Technologies

CLOSED: [2019-11-15 Fri 15:47]

I've been thinking going with Ruby and Sinatra, but it's probably worth looking at other options too.

+I decided to go with Clojure. I think. I might change my mind on that honestly.+ I changed my mind and am going to take a Literate Programming approach with Ruby and Sinatra.

Ruby + Sinatra

Clojure

Workers/Serverless

Using some kind of serverless model might be economical. But would also require me to figure out how to do that.

TODO Find sites to Aggregate

    Ensure that the sites ToS are OK with this and they have like an API.
  • Show HackerNews
  • ProductHunt
  • GitHub trending
  • Reddit?
  • Other?

Learn their APIs and get data

Filter to only relevant

DONE Consider Literate Programming

Using Org-Mode and Org-Babel-Tangle I could write this whole project in a literate config manner. Which might be a very cool idea.

Setup

Rakefile

Rake is the basic build and automation system of Ruby. It is similar in purpose to the classic Make tool, but is Ruby-specific and used for convenience.

The Rakefile has 2 tasks, one installs dependencies and the other runs the project. Both leverage the Ruby Bundler to keep dependencies project-local.


task :deps do
  system "bundle install --path vendor/bundle"
end

task :run  do
  system "bundle exec ruby main.rb"
end

Gemfile

The Gemfile defines the dependencies of a Ruby project. In this case we just use the default file provided by ~bundler init~ and add Sinatra as a dependency.


# frozen_string_literal: true

source "https://rubygems.org"

git_source(:github) {|repo_name| "https://github.com/#{repo_name}" }

gem 'sinatra'

The Gemfile will also generate a Gemfile.lock which containst the exact version information of the dependencies.

main.rb

Imports

The main.rb file is the entrypoint of the program and contains the Sinatra handlers for the various URL paths. This file should be fairly short, as there aren't many URL paths and they are only rendering HTML templates. First we have to import the libraries in use.


require 'sinatra'
require 'erb'

Index / GET Handler

This is the handler for the main index page that has most of the content.


get '/' do
  erb :index
end

About Page /about Handler

It's always nice to have an about page too!


get '/about' do
  erb :about
end

Templates

The templates are the HTML and templates code that get rendered into the final HTML which is served to the user.

Sinatra looks for templates in the views/ folder so we have to make sure these files get tangled there. We are also using Ruby's built-in ERB templating system since that's easy and powerful.

Layout Template

The layout template is what all othe template views will be rendered inside. It's a basic form of template inheritance, and it saves us some typing.

This is the basic structure of the document. The <body> simply contains the ERB code to inline the child templates. We'll get to the <head> down below.


<!DOCTYPE html5>
<html>
  <head>
    <<head>>
  </head>
  <body>
    <%= yield %>
  </body>
</html>

The <head> of the HTML document contains the various meta-information about the document that we want to have on every page including the title and various other attributes.

Most importantly it also has the <link> to the CSS. For this project I have opted to use the Tachyons CSS framework since it is simple and has the features I needed. Also I like how it looks.


<title>NextBigThing.tech</title>
<link rel="stylesheet" type="text/css" href="/static/main.css">
<link rel="stylesheet" href="https://unpkg.com/tachyons@4.10.0/css/tachyons.min.css"/>
<meta charset="UTF-8">
<meta name="description" content="Trending tech and ">
<meta name="keywords" content="trending,tech,technology,aggregator,hackernews,producthunt,github">
<meta name="author" content="Steven vanZyl">
<meta name="viewport" content="width=device-width, initial-scale=1.0">

Index Template


Hello!

About Template


about!

Get the Data

ProductHunt

Show HackerNews

Listing merging

Of course to be an aggregator we need to aggregate some data. So we need to get data from a few different sources. If a project is on multiple sites it should be merged into one listing with a compound popularity.

Listing ranking

Listings should be ranked based on their popularity.

Algorithm for determining rank based on various sites

Different sites need to be weighted differently.

Unified Listing type

A single type for all listings so that they follow a common setup.