Nash's world

Tag: Web


Reddit URL Harvester

by on Mar.31, 2011, under cool, Programing

snooI was browsing Reddit and I stumbled  upon a small sub-reddit at /r/Earthporn this small conner of Reddit had higres pictures of earth scenery.  From there i was directed to  /r/CityPorn, /r/SpacePorn , /r/MachinePorn/r/AnimalPorn and /r/BotanicalPorn all HD pic. This got me thinking since i am lazy as fuck i did not want to spend every day going through six sub-reddit download and save them the best way to do is to automate the process.

In order to download the images the first thing that needed to be down was to harvest the urls from the sub-reddits home page. I could have gone and screen scraped with beautiful soup but reddit provides this nifty feature where by if you append .json to end of a url eg http://www.reddit.com/r/earthporn/.json it will return the json file for the corresponding page with posts and urls and other data (also if you append .xml it will return and XML file with the pages data ). This allows me to skip all the dirty crud of passing html of a constenly changing page. Below is the version 0.0.1 of URL harvester code.
(continue reading…)

Leave a Comment :, , , , , more...



PHP variable variables; “variable variable takes the value of a variable and treats that as the name of a variable”. Also, variable.

by on Oct.19, 2010, under Programing

this is only the entrance of the rabbit hole.

If you understand what this expression really does, you realize that you’re gazing upon the entrance to R’lyeh.Do you think you don’t need your soul anymore? If you do, follow me into the lair of the Elder Gods. But be warned, you will die a lot inside.The first thing to understand is what $ is. $ is actually a shorthand for ${} and means “return the value of the variable whose name is contained in this”.That variable name is a string.

A bare word is a PHP string. Let that sink for a second, because we’ll come back to it later: $foo really is the concatenation of $ the variable-indirection character (think *foo) and foo which is a string. foo === "foo" in PHP, even though in raw source code you’ll probably get a warning. If those are enabled.

(continue reading…)

4 Comments :, , , more...


41 Reasons why I shouldn’t be a father

by on Jun.13, 2010, under Uncategorized

  • 1. I’d undoubtably call the child Spider-man or Optimus Prime or something.
  • 2. Failing that, I’d just call him or her Optimus Prime or Spiderman or something, regardless of what’s on his/her’s birth cert. To prove, I’ll name my hypothetical child Shepard for the remainder of this list.
  • 3. Shepard’s first language would be C.
  • 4. I’d train him from day one to be a superhero some day. This is apparently a bad thing.
  • 5. I’d smack him if he didn’t use grammar more or less correctly in any language he speaks.
  • 6. I’m dyslexic so he would not be able to spell
  • 7. I’d leave him in the wilderness at a young age for a few weeks to let him fend for himself.
  • 8. I’d feed him a small amount of various toxins every few days in increasing amounts to help him build up a resistance.
  • 9. I’d train him in the art of gun kata from four years of age.
  • 10. Shepard’s mother would presumably be some sort of android.
  • 11. Speaking of, at birth, I would replace a few of his body parts with replaceable robotic ones.
  • 12. I’d end up eating all of his Coco Pops.

(continue reading…)

10 Comments :, , , , more...

Strategies to mitigate web application database scalability issues via the use of NOSQL

by on Mar.12, 2010, under APIIT/UCTI

    Introduction

The rise of web applications — websites that replace the functions of a software program that was traditionally installed on a personal computer – was one of the hottest topics in the tech industry. Huge numbers of “Web 2.0″ startups are competing for user attention, and many observers predict rapid growth for web applications (Rubicon Consulting, Inc. 2007). Usage of a web application can outpace initial expectations. Growth is good for business but creates some real challenges when it comes to trying to keep everything up and running speedily along specially increasing traffic to web applications poses great challenges to database servers. End users are becoming more and more sensitive to the quality of the offered services. This requires addressing issues such as pushing quality of service (QoS) requirements into database processing and providing database system scalability (Ye 2002). But applications suffer from unpredictable load, especially due to events such as breaking news (e.g., Hurricane Katrina) and sudden popularity spikes (e.g., the “Slashdot Effect”) (Amit Manjhi 2009). Investing in a server farm that can accommodate such high loads is not only expensive (particularly after factoring in the management costs) but also risky because the expected customers might not show up. Content Delivery Networks (CDNs) provide such service by maintaining a large, shared infrastructure to absorb the load spikes that may occur for any individual application. However, CDNs currently do not provide a way to scale the database component of a Web application they only provide a way to mitigate network load hence the CDN solution is not sufficient when the database system is the bottle-neck, as such is in many web applications.

(continue reading…)

11 Comments :, , , more...

Facebook has rewritten the PHP runtime from scratch

by on Jan.31, 2010, under Programing

It seems the rumors that php being rewritten by facebook seems true. A new open source project that is going to be released by them come this tuesday.  The unofficial word is that it is some sort of compiler for php and some poor soul has been hacking away trying to implement it for past one year and it is rumored to have reduced the cpu usage by 80%. php stuff This is pretty significant numbers if its true. For most of the webapps the performance issues are going to be on the network latency and DB rather than the script engine but with a 80% reduction is cpu time means the page can be chuged out 400% faster dramatically reducing the hardware costs.  with user numbers in 100’s of millions for facebook even 1% gain in performance will save them a chunk of change.

Every one of you know how much a evagilist i am when it comes to python while no where fast as .net or Java it can pretty much wipe the floor with every day venilla php in performance numbers, especially with Mod_wsgi or fast_cgi implemented. Facebook with thousands of man hours invested in their code base they cannot just up and switch to another language with out major cost or headaches. So they have done the next best thing changed the internals of the tired old donkey they being riding around so that its rocket powered so that it can run with the Lamborghini’s.

Leave a Comment :, , more...

single equation for the graph of a marijuana leaf

by on Jan.25, 2010, under Art, cool

here is a cool poler plot graph that draws out a pot leaf
PolarPlot[(1 + 0.9 Cos[8 t]) (1 + 0.1 Cos[24 t]) (0.9 + 0.05 Cos[200 t]) (1 + Sin[t]), {t, -Pi, Pi}]


the out put of the equation is below

click here to see the the equation plotted by wolfram alpha

3 Comments :, , , , , more...

Looking for something?

Use the form below to search the site:

Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!

Blogroll

A few highly recommended websites...

IDL

Member of The Internet Defense League