Ec2 Videos

This is part 4 of 6 of the Demystifying the Cloud series. This episode provides an overview of Amazon Web Services and the scenarios that target AWS.

We often get the opportunity to help out our fellow starts-ups with their servers. While being a start-up with scaling issues is a sign that things are going well, sometimes such a small team does not have the expertise to make sure all their servers are in order. We wanted to share a couple pitfalls that we have helped diagnose in hopes to prevent other start-ups from doing the same thing. 1. Switching web servers when in trouble All the sudden your web server has a 10.0 load average and the Apache process is hogging memory and pegging the CPU. What do you do? Switch out Apache for nginx, of course. Just kidding! Although, you would be surprised how often the web server gets swapped out during a time of peril. Under extreme loads (such as benchmarks) some web servers out perform others in subtle ways. However, under normal growing start-up load, slow downs are more likely due to misconfiguration or poorly written application code. If you switch to another web server, you might luck out and get a better default configuration, but that is about it. A few things to keep in mind while administering a web server: Swap is a killer. If you hit swap, your entire machine (including your webserver) will slow down dramatically. To diagnose what is going on: run top, press capital M (on current Ubuntu/Debian), and you will get a list of the top memory users. If your webserver is using more than 20-40MB RSS per process, you probably have something configured incorrectly (too many modules loaded, wacky app code, etc). Memory usage of an web server is just a math equation. If you are using Apache with the prefork MPM, each connection requires a new Apache process. If you have something like PHP loaded up in each of those processes, each one will come in around 10-20MB. This means you could only sustain around 100 concurrent connections before running out of memory on a 1GB machine. To get around this particular example, use the worker MPM with Apache, which adds about 1MB overhead per connection via threading. Serving static content is the easiest possible task for any web server. Doing a reverse proxy from Apache to lighttpd (for example) for just static content will get you nothing but complexity that is unless you have Apache misconfiguration to serve static content through your framework. Do not offload your static content unless it is to get load off your application server such as by using a reverse proxy cache. Check your error log! Error logs are a great way to see what is going wrong, yet often go overlooked. 2. Using sqlite in production Many frameworks, including Django and Rails, make it easy to use sqlite as a test database. However, sqlite should never be used in production. It is important to remember that sqlite is single flat file, which means any operation requires a global lock. Locks will inevitably cause points of contention if the database gets even remotely busy. On top of that, your web server will appear to peg the CPU when under load. This is because of all that contention around the lock. The moral of the story is to use mysql or postgres in production. These relational databases are heavily optimized for production environments, and will make your app much more responsive. 3. Forking in app code Sometimes the first pass at a webapp does some crazy stuff. This crazy stuff might involve system calls, such as renaming a file, or checking the last modified date of an image. While issuing system calls per web request might not be the best idea in the first place, under no circumstances should you fork. If you fork inside an app server, such as mod_python, you will fork the entire parent process (apache!). This could happen by calling something like os.system( mv foo bar ) from a python application. It is important to remember that os.system uses the system libc function, meaning that it forks and passes the args to your default shell. Overlooking the security implications of this (of which there are many), this causes big performance problems. Fork is one of the most expensive system calls, and should never be used on a per request basis. Moral here: If you have to use system calls (which you should try not to anyway), never use use fork. Use the native stuff, like (in python) os.rename, os.stat, etc. Hopefully sharing these examples keep future projects from running into scaling trouble before they need to. If you ever have any questions about this or just running servers in general, feel free to email support@cloudkick.com. This post comes from Alex Polvi (@polvi) from CloudKick.  CloudKick is a Rackspace partner and a free way to manage your infrastructure across the Rackspace Cloud, Slicehost, or EC2.  Check out their blog for more info as well as a project Alex is working on called LibCloud, a client library that just made writing cloud apps in Python a lot easier.  If you have Python friends, be sure to mention this project to them because you might make their day.

Pirate Bay Spawns More Sites Lists Widgets for Twitter EC2 Cracking PGP Passwords SDR News is a Daily (M-F) Technology Podcast with Tech News Highlights from Slashdot, Digg and Reddit Click Here to Sign Up for the SDR Newsletter Prefer a Direct Download ? (mp3) Download today s show. SDR News Links If a news item has disappeared from the Del.icio.us list above, try the full list here. SEARCH ANY STORY YOU HEAR ON THE PODCAST More Info on Screencasts Online Apple TV is a Natural Play by Andrew McCaskey The Power in the ITunes Music Store There has got to be some credibility in the multiple rumors that are floating concerning the pitch from Apple to various television networks and content owners to allow Apple to provide television content for a monthly $30-$40 fee through iTunes. Disney seems to be the most likely content provider to buy into the deal, based on the close relationship that Disney and Pixar have had over the years. If there is one thing that Apple has been able to do, it is to show how to effectively sell and deliver digital content efficiently. And with close to 100million installations of the iTunes Music Store software on Windows machines, it has the penetration and delivery depth that even Hulu will envy. One player that will be a lot more difficult to grab will be NBC Universal, especially if the proposed merger with Comcast goes into effect. That could put NBC at a significant dis-advantage with competitive material available on the iTunes platform and an expectation that even more programming be available online in some form. It didn t have to be that way reasonable channel packages from cable and satellite providers that got rid of the shopper channels, television preachers, infomercial channels and similar dreck at $85 a month and moved to that $35 price point could have prevented the whole problem. Apple just sees a consumer need and prepared a product that people will want at a price they are more than willing to pay. Nothing evil about that. Contact Us How to Build Twitter Lists Watch Today s Video Episode Be sure to check out GoToMeeting. Why? Because you can hold meetings right over the Net from anywhere. Plus, you can hold all the meetings you want for one flat rate. To get your free 30-day trial , visit www.gotomeeting.com/techpodcasts. Take 10% off any order at GoDaddy.com! Code SLASH Take $5 off any $30 order at GoDaddy.com! Code SDR530 .com Domains $6.95 at GoDaddy.com! Code SLASH3 More blogs about SlashdotReview and SDRNews

Amazon AWS Case Study by: Dudi Landau , Thomson Reuters", "ClearForest CTO

"Amazon AWS Case Study" By: Dudi Landau , Thomson Reuters", "ClearForest CTO

MySpace on the Cliff Edge Amazon EC2 Not Immune to DDOS Wave: The Eternal Conference Call SDR News is a Daily (M-F) Technology Podcast with Tech News Highlights from Slashdot, Digg and Reddit Click Here to Sign Up for the SDR Newsletter Prefer a Direct Download ? (mp3) Download today s show. SDR News Links If a news item has disappeared from the Del.icio.us list above, try the full list here. SEARCH ANY STORY YOU HEAR ON THE PODCAST More Info on Screencasts Online End of Google Wave: The Eternal Conference Call by Andrew McCaskey That Will Finish Off Google Wave Nickolas Carr noted the ebb and flow of business communications over the past thirty five or forty years, and sets up the nightmare vision that could finish off Google Wave before the seond round of invites hits the streets. One of the advantages of email, when first introduced was the fact that you had much of the immediacy of the telephone but allowed a few minutes to compose the thoughts before the response got sent out. By destroying the synchronous nature of query / response by phone, but still taking advantage of the electronic medium the best of both worlds seemed at hand. It was still attractive, even if the CC and Reply to all allowed duplication with almost no effort. That minimal duplication cost and the ability to substitute activity for thought gave rise to spammers in the public sense, and the far more insidious internal corporate spam that copied everyone on trivia, and set off additional spasms of traffic asking to be taken off the list. Which worked well until the next rising tide of traffic as a result of not being read into a situation. Nickolas notes that Google Wave is nothing but the worst of both worlds: The interruptive immediacy of the telephone, the party-line noise of multiple chattering updates, and the quick to observe notes and phrasing as the words are being typed in. A Brave New World of Corporate communications, indeed. Contact Us HP SuperDome Tech Day Cell Module Detail Watch Today s Video Episode Be sure to check out GoToMeeting. Why? Because you can hold meetings right over the Net from anywhere. Plus, you can hold all the meetings you want for one flat rate. To get your free 30-day trial , visit www.gotomeeting.com/techpodcasts. Take 10% off any order at GoDaddy.com! Code SLASH Take $5 off any $30 order at GoDaddy.com! Code SDR530 .com Domains $6.95 at GoDaddy.com! Code SLASH3 More blogs about SlashdotReview and SDRNews