What will you be doing with it? Is it for taking everywhere with you or is it just for special events and that sort of thing?
Things you can think about:
memory type: there are several different card types. i use an SD card in my camera (pentax optio S) because i have an SD port on my laptop. i've heard good things about XD (i think it's XD..) cards as well - but they tend to be in the upper range cameras.
note that you can usually change the memory card (dont get a camera that wont let you do this!) to get more memory, or to swap cards if you run out of space on one. you can't change card TYPES though since they're different sizes and use different technology. card sizes vary and how many images you can store varies (see resolution below)
battery type: there's generally two types you can get, normal AA batteries or the camera's own battery. there are pros and cons of both:
AA batteries: don't last long, but can be replaced really easily, or you can get rechargeables
camera batteries (like li-ion ones i guess): last longer, but cost more to replace when they do die. also some cameras require the camera to be plugged into the charger / cradle so you can't do anything with it in the meantime..
zoom: two types of zoom.
normal zoom (sorry, i have no idea what the real terms for half these things are, hehe) is defined like a typical analogue camera zoom.
optical zoom is (this is how i see it) an additional zoom level on top of normal zoom. rather than using the lens to zoom it actually enlarges the pixels of the image. if you zoom in all the way with optical zoom it tends to make the picture go a little fuzzy or blurry.
most cameras under $300 will have about 3-3.5 megapixels, which is more than enough for happy snaps. someone with more digital camera knowledge can probably explain this better than me :S
image resolution
different resolution sizes (usually related to the # of MPs ?) tell you how big your images can be.
high res - ie. 1024x768 and larger: good for printing, better quality, can be resized without blurriness or image loss,
low res 800x600 and smaller: good for emailing or uploading online, sending to friends etc. it's just a basic image which can be resized for avatars and that sort of thing, but doesn't look too good printed and can't be made LARGER.
something which really concerned me when i bought my camera was the speed with which it can take pictures (shutter speed) ... i like being able to take an almost instantaneous picture. there's a few of those point-and-shoot cameras around and they're good for when you go out with friends etc.
flash
some digicams have a really bad flash, so make sure you check that it can take night pictures. esp for the lower range of cameras where flash is not a priority.
other things..
brands which are GOOD are nikon, pentax, canon, sony is ok but expensive. and i cant think of any others hmmm...
do a google search online to find reviews if you think you find a camera you like.
and make sure it will last you