domenica 20 novembre 2016

Bitbucket or Github + Hostmonster + MAMP auto deployment and clone

In OSX termina type in:

xcode-select --install
cd /Applications/MAMP/htdocs/
git init
git remote add origin ssh://git@bitbucket.org/musicstat/musicstat.git

Once you created your local files and put in your MAMP folder, login in Bitbucket and add your OSX ssh key for your user:
  • check that your ssh agent is running ps -e | grep [s]sh-agent
  • if not, start it ssh-agent /bin/bash
  • show your ssh key cat ~/.ssh/id_rsa.pub
  • if empty, you don't have a default key and need to generate it ssh-keygen
  • on OSX, copy the key into the clipboard pbcopy < ~/.ssh/id_rsa.pub
  • go to https://bitbucket.org/account/user/youruser -> SECURITY ssh keys
  • paste and don't remove anything, a mail will be sent by Bitbucket

git add --all


git commit -m 'initial comment'

git push -u origin master



This allows to have a local MAMP repository and a remote Bitbucket repository. Hostmonster should autoalign deploying from Bitbucket: as soon as you commit your local branch, Bitbucket updates and will update Hostmonster accordingly.

For the Hostmonster part, download the files here and upload in your website (/deploy folder):

https://bitbucket.org/lilliputten/automatic-bitbucket-deploy/src

Connect to your ssh Hostmonster hosting, go to the folder to be aligned:

pwd returns a full path e.g. 

/home2/youruser/public_html/yourwebsite/test

This path is the one to write in test/deploy/deploy-config.php
Once done, you need to setup www.yourwebsite.com/test/deploy as a "webhook" url inside Bitbucket: once BB receives a commit for new code from your local MAMP repo, visits the URL on your website, which will automatically git clone the new code:
  • Go to Bitbucket SERVICES (bottom menu item) -> Webhooks -> Add Webhook
  • fill in details, as default the action is bound on trigger "repository push", but you can also specify additional ones
The path would be like this: yourwebsite/test/deploy/bitbucket.php
or yourwebsite/test/deploy/github.php

The next time you will change something inside MAMP, you just go in MAMP folder and then

git add -A && git commit -m "modifications comment" && git push -u origin master

martedì 1 novembre 2016

Why the music streaming business model is undermined (& how to build a Clicker-as-a-Service)

INCIPIT

2015 was the year of the botnets.
2016 was the year of the botnets business model.

(Ro)bots are devices like PCs, webcams, virtually anything with an operating system able to host a code, to wait for a command sent remotely, to execute it.

Botnets are networks made of such bots, tied together by a central authority, better known as "C&C command & control", that sends them the remote commands to be executed.

Typically the cybercrime industries (yes, hackers belong to past) send a malware via email to PC users, who click on it and unknowingly transform their PC into a "zombie", a bot ready to receive hidden commands by the C&C. But organizations are securing their networks, awareness is increasing among users, operative systems are including even more security checks to fight against advanced threats. For this reason, recently the cybercrime companies are better targeting IoT devices (webcams, wind turbines, industrial machines, intelligent household appliances, etc) because they have simple and very specialized operative system, with almost no security by design, connected on barely secured networks. IT and Engineering offices inside a company typically struggle to cooperate and Engineering office doesn't allocate budget to get a modern IT network security, considered "non-core" for the machine-to-machine communications.

This obtusity brought us to the most powerful DDoS attack in the history: on 21.10.2016 DYN DNS experienced a 1.2 Tbps DDos attack, performed by a botnet mostly made of IoT devices. To simplify, imagine billions zombies simultaneous opening the same webpage over and over: the server bandwidth will be exausted in seconds. In addition, an innocent outbound connection to a website will pass unobserved through any security check.

Several cybercrime companies also made Cloud Services which leverage these botnets to perform DDoS, send spam and viruses. Do you need to hit hard your competitor's network or website? Just google around for dozen Cybercrime Cloud Service providers, choose the cheapest, pay a bitcoin and you're on the go. Simple & funky.

Cybercrime business model made easy... just a couple months to write some PHP / MySQL code to build an admin control panel, an user portal, a bitcoin shopping cart. Bash code and botnet workforce automatically do the rest. You can object that building a botnet requires skills and time, but perhaps you don't know Shodan search engine, free botnet software (google yourself for "botnet github"). And building a crawler for exposed services with default passwords is even a trivial task in bash, python, php or whatever language you're comfortable with. In addition the IPv6 addressing era is next to come, so you can expect your washing machine to become the perfect zombie: intelligent operative system, public connection, no perimetric neither onboard defense. Cheers!


A BOTNET FOR MUSIC STREAMING SERVICES

Well, conceptually speaking this Cybercrime-as-a-Service business model can apply to any music streaming service: it can be a "Clicker-as-a-Service". Imagine bots generating clicks on your song... you could get million clicks a day, chart positions, fame, and obviously money.

Prof. Pelle Snickars at Humlab (Umea University, Sweden) is talking about the weakness of streaming services business model since 2015 and made a clicker himself: http://pellesnickars.se/wordpress/wp-content/uploads/2016/07/snickars_talk_DH16.pdf

Humlab built a Spotify clicker written with selenium webdriver, which perfectly fits the Botnet-as-a-Service business model:
  • it's headless, so nobody can imagine to have it resident on his device
  • it can be hidden behind a proxy, so it can simulate multiple simultaneous countries
  • it can simulate any browser
They called it Spotibot, like the bot in the botnet, huh? :-)


HOW TO BUILD THE CLICKER BOT

Won't linger on other conceptual ways the music streaming business model can be crumbled, because I already spent months, money, effort to elevate the Humlab concept to steroids, over surpassing their Spotibot features. Perhaps I will keep this task for the next year.

My concept worked, except that I deliberately fenced its action on my PC, using only 4 real accounts and my own songs, in order to inspect the Spotify BETA (and delayed and bugged...) distributors portal for the results. So limited, it was still capable to generate 300 clicks (about 10€) / day (depending on songs length and Spotify calculations) ) with just 4 accounts running 24H a couple days. Thus minimizing the impact on Spotify policies, but at the same time undoubtedly proving that the concept do works. And you need to prove the sickness to master the cure, but this is a matter for other tables of discussion.

The challenge was initially approached making use of phantomjs, another headless browser, that soon revealed its weakness towards the Spotify web player. Spotify is completely based on RTMP streaming technology and makes use of the infamous Flash Player plugin. Phantomjs would have worked with other music streaming services, that are based on HTML5 players instead, like e.g. Youtube.

Selenium webdriver here is the best choice because mimics the user behavior in a real operative system, with the ability to run headlessly (the graphical user interface is loaded inside the RAM). In addition it can help in humanizing interactions and it can be visually inspected with VNC at runtime e.g. to solve captchas, to debug, to interact with, and other advantages out of scope.

On the connection side, we can seamlessly leverage the native ability for Mozilla browsers (Firefox, Chrome) to create segregated environments just inside any browser window. I mean: if Firefox opens 10 windows connecting to the same website, every window has an independent IP address from 10 independent countries, independent useragents and cookies, independent browser sessions with 10 different login credentials, thus emulating 10 independent real people. You can multiply 10 * any number, until you have enough money to buy cheap shared proxies plans and CPU power.

RAM is not an issue, CPU is. The reason stands on the old buggy Flash player, crashing from time to time, expecially when the CPU load exceeds 75%. The solution has a complex implementation, but can be summarized with monitoring Flash inside selenium itself and reloading the window if the script stops detecting Flash. The only annoyance is the need to manually solve the captcha (an email alert is generated), unless you decide to implement a cheap captcha solving service (deathbycaptcha, etc), with affordable but unpredictable costs that depend on crashes rate.

A couple bash scripts take charge of system updates and startup after reboot.


HOW TO BUILD THE CLOUD SERVICE

Now that you have the clicker bot, you need to setup the cloud service. The bot can retrieve accounts, associated proxies, useragents, eventual humanizations and other commands, from a remote MySQL database. Also the cloud service and the admin control panel lay on the same database. Other tasks and web interfaces are easily built in PHP / MySQL rather than other languages.

I forgot to mention that you'd better prepare a script for user registration (it's trivial on Spotify even using e.g. PHP + cURL) and another one to add / remove tracks to playlists. You can go really further with humanizations, scrambling, simulations, auto email verification if needed. But once again this is out of scope here. 
For building the database, you just need a users table for your customers. Another table is needed to store registered accounts and give them as many details as required in the registration form, an associated proxy and useragent in order to always present them with the same IP address, browser, OS details. I could really push this concept further with joined tables: real users generation, real addresses generation, useragent autoupdate and other refinements. I could just reused several scripts from my personal developer's vault, but again it's out of scope here, just a pill of inspiration for you folks.

The user's control panel just has a shopping cart code, but I blame the countless free public codes around because they can lead to exploitable vulnerabilities. When money is key, I rather prefer a manual approach: a transactions database is quite a trivial task if you're familiar with PHP/MySQL. Email alerting is on charge of the payments provider, that will send you an email notification upon receiving the money.

You can even provide the possibility for the user to add his own Spotify accounts and passwords, so leveraging the user for any possible policy break. You made the knife, then it's 3rd party responsibility for its fair use, e.g. for slicing food.


CONCLUSIONS

Having all in place and working, this clicker bot can be installed for thousand times on thousand machines (hence the botnet we talked before). It takes a few months, some money invested on specialized freelance developers to build some code you couldn't manage yourself, some creativity to solve some technical challenges and system optimizations. In addition, there are some recurring costs for proxies and infrastructures, but a raw ROI calculation based on my tests without indirect costs can suggest a gross break-even in less than a year:

  • 300.000 generate 100€ payout
  • 10 simultaneous accounts, clicking on a track with a duration of about 4 minutes, generate about 2500 clicks/day (25€/month)
  • An retail inexpensive PC (dual CPU, 4GB RAM) can afford up to 20 simultaneous accounts, so doubling the clicks. Halving your song duration to 2,5 minutes enables quadruplication of clicks (and revenues): 100€/month
You can mix your own tracks to the user's tracks e.g. 3 own tracks every 7 user's tracks (525€ / month). If you skip building a service cloud and use your customers' infrastructures as multiply factor to generate clicks for your own song, you come up with a robbery in spite of genuine clicks...
I would like you to jump to conclusions yourself, but these seem quite clear to me: the streaming business model is weak the way it is, exactly as suggested by Prof. Snickars.