Dude, That's Just How I Roll
Through this I worked on a lot of ideas and statistics about what different dice mechanics feel like. The mechanic chosen can really affect the feel of the game, so picking the right one is very important. Some lend themselves better to gritty games, where death is cheap and fast, while others work best for big scale cinematic style games. The number of dice you want to require your players to have is also important factor, but nowhere near as important as the actual mechanic's feel.
In general there are three basic kinds of Dice Mechanics, although each one has several variations. The three basic styles are a Flat Die Roll, a Bell Curve, and the Die Pool. I will do my best to discuss the basics of each one here and little about how they can be used. I may also get into a few alternatives to dice, but this is mainly going to be about dice roll types.
The most commonly used Flat Die rolls used in game are the D20 and the D100 (or percentile dice). Since these are a single die with an even chance of every side coming up, the odds of any particular number is equal to the number of sides on the die. So a D20 has a 1 in 20 (or 5%) chance of any particular number being rolled and the D100 has a 1 in 100 (or 1%) chance. These odds do not change unless you change the die used, but then it just changes to another flat range.
This can work great for systems where the die is rolled against different numbers based on the difficulty of what is being attempted (such as skill level number or a challenge number of some kind). There is often something added (or subtracted) from the roll based on the situation or the skills of the character, but the real difficulty of something is set by altering that target number.
For example: A system may use a D20+Skill system (where they roll D20 and add what their character has for a skill level in what they are attempting) and they roll against the Difficulty Rating (DR) set by the GM based on the situation. Rolling a total over the DR means a success. The easier the thing being attempted the lower the DR.
This can also be used well when the roll is used against an equal roll from an opponent, usually with both adding some kind of skill or bonuses to the outcomes. The opponent can either be another character (as is common with combat) or the GM rolling for some other challenge element. This can allow for a wider range of outcomes than a single roll, while still allowing one side to have a greater advantage (if they have the skills or bonuses to achieve as much).
The upside of using a flat roll is that it makes selecting target difficulty numbers, skill levels, and bonuses much simpler on the game designer or GM, since the range of numbers and odds of those numbers coming up are very static. It also makes opposing roll systems rely a lot more on luck than skill, which can make for an exciting game.
The downside of a flat roll system is the lack of variation between rolls. Every number is exactly the same odds as every other number, so all diversity is completely dependant on the skill numbers, target numbers, and bonuses. This can make some games feel very static or, well, flat.
In the middle of it all are some aspects that can be considered either good or bad, depending on the feel you are going for. In games with critical rolls (meaning automatic success or failure) the odds of these happening are usually more common, making them less meaningful but possibly making for more exciting games (depending on what feel you want for the game).
Let me give some examples here.
As you can see, curve becomes far more extreme (and far more likely to hit the central numbers) the more dice you add, and the far less likely the fringe numbers are to hit. I also added an example of what you get if you roll 4D6 and remove the lowest roll before adding the top 3 dice together. This give a skewed bell curve that favors the high numbers, but still lands them closer to 13 more often than either extreme.
Using a bell curve style of roll will make most rolls fall in the midrange and allow the extremely high and low rolls to be special rarities. This can allow for far more rare criticals and make high and low rolls feel more exciting, but also tends to make most rolls feel more average. This is usually a more realistic (if not more exciting) way of handling things, especially when still using the roll to go against some kind of target number.
The more dice involved in the roll the more extreme the effects of modifiers become, as the rolls become more centralized on average. Adding any numbers to the roll pushes the effects higher, altering the bell curve only by sliding it up by that number. Adding dice to the roll as a modifier will not only cause the center of the curve to move up slightly, but also increase the size of the whole curve.
The upside of the bell curve style of roll is that it tends to favor a certain smaller range of numbers, allowing things outside the norm to become more rare and special. This is more of a realistic way of handling events, in most cases, and makes modifiers far more important to use.
The downside is that criticals (both good and bad) become more rare the more dice are involved in the curve, which leads to more actions being average. This is something that many gamers find to be boring since they like the excitement of larger range of numbers.
There are two main ways that die pool games tend to use this method. One is simply to add all the numbers together and compare them to some target number (or opponent's roll). The only reason this differs from the bell curve method is that there is no set number of dice being rolled. The more dice added to the pool, the higher that central part of the curve gets, but the extremes become harder to reach. Still, the outcome on average will always be higher with more dice.
The other method is to treat each die separately, comparing them to a target number, and counting the number of successes (and possibly failures). This method ends up with a much smaller number (which is easier to handle within most rule systems) but each added die can really affect the possible outcome because of this. Things can also be greatly altered by changing what is determined a success per die. This can lead to some really exciting happenings when they all come up one way or the other.
For example: A character with a die pool of 7 dice (where the range tends to be between 3 and 10) has a chance of getting a possible 7 successes, which would be an amazing success for the system. Lets say there is a 1 in four chance of success per die, so the odds of that would be very slim (and the average number of successes would be just below 2 for this character), but the fact that it is possible makes for an exciting risk.
Quite often dice pools combine multiple things to find out what is rolled, such as adding the number of dice for an attribute to the number of dice for the skill being attempted. This can quickly add up if people pump up certain skills and attributes, allowing for certain things to be amazingly successful most of the time. This is where the "Bucket of Dice" thing comes into play. I have seen some systems where people re rolling dozens of dice just for one action.
The upside is that it is actually a lot of fun to roll dice, so the act of pouring a handful of dice out to make some action can be a blast. This is also a good way to actually make different aspects of a game very meaningful (like attribute skill numbers). Plus it makes such a cool sound. This can also easily be used to quickly show how skillful somebody is without having to check their sheet at all. You see how many dice they roll and know how good or bad they are.
The downside is that it can be a real pain in the ass to have to count up large numbers of dice. Counting successes is easier than adding up all the dice, but can still take a while. Your opponents will also be able to see how good or bad you are at something by the number of dice you roll. Since there is no set number of dice that are rolled, it is often hard to include some kind of critical roll mechanic into the game.
At a future time I may get into what to do with your dice mechanic in your system once you decide on one, but I think this primer does a good job of covering the basic differences between the three methods of generating numbers. I hope you found this helpful and enjoyable. As usual, if you have any comments or suggestions feel free to leave them here or in the forums.