There are many tools for learning programming, and most are available for free. These tools vary in complexity, with drag-and-drop tools being a very good starting place and allowing users to progress onto text-based programming such as Python.
Scratch is a very useful tool for those new to programming. Its simple, drag-and-drop interface makes it perfect for those completely new to programming. With a simple sprite-based system and a variety of instructions, it can be used to make animations, 2D games and other basic programs.
Scratch is often taught in primary schools as an easy way of learning programming - it is generally taught from years 5-7, and generally progresses onto more complicated languages.
By the same team who created Scratch, AppInventor is an online tool with a very similar interface which can be used to make Android apps. It can create 2D games and utilities, but the testing system & resulting apps can be quite slow.
Learning to create hardware devices - ranging from something as basic as an LED to a complex game with a microcontroller - can, for many people, be considerably harder than learning programming. However, it can still be learnt with relative ease if the right tools are used.
To start learning about electronics and basic circuits, you can get a starter kit - a good place for electronics is Maplin. For example, you can make a basic LED circuit with some cheap components - the most important being a breadboard, which allows for components and circuits to be tested easily.
A microcontroller is a small device with a low-power processor in it. This allows it to perform basic computation operations with ease whilst being relatively low-power; they are often powered via USB. They have electronic pins which allow them to take inputs and give outputs.
Arduino is a well-known brand of microcontrollers. They have many code samples for various tasks built into their programming IDE.
Official Arduinos can be somewhat expensive; however, on sites such as Amazon, there are cheap 'clones' of the product, which are almost always functionally equivalent.
There are many other brands of microcontroller, but Arduinos are aimed at beginners, and are very well documented. This makes them very accessible to people new to electronics.