We have just noticed that a Turkish company already being called Taba Media.
Hence this is the last post under the name of Taba Media Soultions, from now on we are TabaServe.
You can find us under tabaserve.wordpress.com.
All the best!
TabServe
Latest Entries »
This is the latest about Twitter, looks like they started to crack down on those who jams up their infrastructure with useless junk advertisements. Apparently their legal efforts will be focused on those who distribute application capable to provide a simple spamming services on Twitter.
I guess the message is pretty simple here, mess with Twitter business processes and you’ll be summoned to court in no time!
Altogether, spamming is pretty pointless, I don’t get why anyone waste any resource doing that…
As the title says today, I am in the process of creating an over-arching social media tool involving some java-middleware, JavaDB and the Twitter API. The API will initially be used via JTwitter written by a Cambridge doctorate guy – initial test of the library were good yet I noticed some very minor leak – will see how it will work on a long run.
In order to save memory footprint on the cloud server the system will be running on CACAO JVM with embedded Java-DB.
If the system is up and running I will post further stuff here.
Just a brief catch-up on other things, got a cloud service account from Glesys – a swedish cloud hosting company, they are cheap and everything looks stable for now.
Well, this is all for now.
I started to look thorught one of the datasets related to a sub-sector, which will be presented on my site one day and tried to find out what to do with it in order to make it more useable. At a glance I pinpointed several major problems, such as redundancy, near-duplicate names and the rest.
So at first I decided to create a small Java application which goes thorugh with a for loop on the database line-by-line, takes the Strings out from a ResultSet then calls a SELECT statement with the WHERE clause, where the previously received Strings are used for filtering the results in the ResultSet. Where the received ResultSet had more than 1 element it all get marked up as duplicates in the datatable.
In order to know how the duplicates are related – as these are not complete duplicates, only certain elements are, the found duplicates are marked up by the PK id number of the original, non-dup elements received a null value. The startegy here is that these duplicates will be added to a separate database and linked back to the main datbase using the PK of the original value with a many-to-one manner…make sense?
Needless to say I completely forgot to add index to my data table before doing this work and it took me a few minutes to realize that something is not right:) So, a word of advice, before doing anything with your database, INDEX THE NECESSARY COLUMNS!
If you are trying to enter into the realms of web mining you are getting dangerously close to the grey word of data scraping, which might cause you some slight headaches in a form of a jolly webmaster. Obviously you got more chances to be flamed out a bit if you don’t follow the robot.txt requirements and your app start squeezing every bit of resource out from a server still it is often a good practice to hide your backside behind some proxies. Those are not cheap though, so he next step here is to get a decent list of free proxies. Then the problem start to escalate….
You might find a long list of free anonym proxies, filter those through, find out that 90% is useless already, got the rest into your app database, let the app start using that list when you realize that after a few hours of work they start falling out slowly but surely.
I mean yes, I thought about that hence I put down some defensive code in the `catch` part yet I never thought that there will be 20 of proxies falling out in the same minute from different geolocations - so what I should have been done is `catching` within `catch` or just design my app slightly better, letting the catch part simply re-initiate the socket opening ceremony with a new proxy.
Once again, free proxies are good but be prepared if they start to fall out in big numbers.
As I managed to miss out the advanced database design module from my studies these days I am required to dig down into the realms of such a topic. As usual, I didn’t know where to start, when I come across with the book called “Beginning Database Design” from 2006. The author is called Gavin Powell and the book teaches you the fundamentals on how to design your first proper relational database.
The major advantage of this book is that not db-engine specific, so you can read it as a proper University textbook and what you get out from there it can be taken away to any platform or industry alike.
Right, I am back to work as usual – yes, even at Christmas I work, I am being overly serious these days. Well, enjoy the Christmas and happy new year to everyone.
Apparently I need to build up my own server if I don’t want to sign up for any cloud service at the moment, so I am happy to pick up any old non-wanted computer for free from your house. Please drop me a line on tamasdev[sign]mail[dot]com and we can sort out the rest.
PS: This is a free service, I will not take any money for computer disposal.
Well, I still couldn’t find the right sort of solution for this problem, I can just assume that it is either to do something with an in-built, hardware level sleep function or a defunct Linux-driver but most probably it is just a bug in the Ubuntu, so it locks up the laptop after a while.
I think it is the first option is the most likely, if I touch the buttons on a daily basis, the system keeps running. Ubuntu is – usually – bug free and there isn’t anything remotely unusual hardware device-wise in the laptop that would justify the use of a non-standard ubuntu-driver. ACPI check showed that all of those are disabled so they shouldn’t be the cause of the problem either.
So, the outcome of this investigation is to stop the investigation as I can spend months on this ’till I found the solution – which is not acceptable really. So I got back into web mining and tapping the laptop’s keyboard regularly. Job’s done…
Yes this is not something what you want to see, especially when your multi-threaded miner running on this. I’ve got Ubuntu 10.04.3 (forgot its codename) on an old laptop, also I got the laptop-mode-tools running on the platform. I can switch the screen off and it works ok for a few days then it cuts out, can’t connect via ssh and the whole system is just “goes to sleep”. If I re-start the screen it wakes up again.
Rather annoying problem, looks like I need to get involved on some Ubuntu-forum to get some ideas for the solution.
Mind you if you got any ideas please share with me as a comment, I’d be grateful…
…no money involved though:)
Soultion:
Probably to do with the “sleep-mode”? Will give it a try and let you know what happened.
I personally not much of an SQL-guru, obviously it was part of my undergard computing studies and I do it regulary when my application needs to communicate with a database via JDBC (I am not much of a big fun of third party-frameworks like Hibernate or Persistence, not because I think they are not good or something, I don’t really know Hibernate and JPA seemed to me as something that I actually don’t need in any of my applications).
Anyway, today’s story is about learning the use – or more like gaining practical expertise – of SQL via MySQL workbench. It is a pretty handy tool and if you managed to get the MySQL instance running properly on your system and connect it to the workbench then right-click on a data-table will give you an option to “Send to SQL editor”. Under this there are various basic SQL constructs that can be sent to the editor bit. You should see something like this:
If you select one of those statements then it will be sent to the editor, where you can further tailor it to your need. I think this is an excellent way of gaining some practical expertise with SQL – build your database and start working on it. Simple…:)
