When I looked into the API documentation page of hook_field_info, it seems to be quite complete. But I could not see the complete details of the return array ingredients.
Why cannot I find the allowed values and detailed descriptions of the following values? Did I miss any documentation page that explain them more in detail?
The only documentation you find about those arguments is the one you already linked (hook_field_info()).
The reason that documentation is not very specific is that each field implemented from a module can use any value it needs. For example, the Image field implemented from the Image module uses the following keys for the settings (see image_field_info()):
Vice versa, the Decimal field implemented from the Number module uses the following keys (see number_field_info()):
The default formatter, and the default widget are (respectively) the machine name used for the default formatter, and the machine name of the default widget. The different between formatters, and widgets is that a formatter shows the value of the field, while a widget requests the values necessary for a field.
If you want to see which values are used from a field, you need to check the implementations of the following hooks done from the module implementing the field:
Really, the values in settings and instance_settings are not set in stone; they will contain keys for settings specific to that particular field type.
They're arbitrary variables whose values will be used as the default for the field settings form. Exactly the same goes for instance_settings.
When you visit a page to edit settings for a field (e.g. admin/structure/types/manage/page/fields/body) you'll see 2 fieldsets. One (the top one) is the instance settings and the other is the field settings.
The default values for the instance settings are garnered from instance_settings on that field type's hook_field_info() implementation, and the same goes for the field settings.
The only way to find out what each one of the pre-defined field settings names is, is to read through all field-providing modules. I can't imagine that information having been summarised anywhere as it wouldn't really make sense to do so.
For the specific example you gave, the authors of the text field module decided that there was a setting required for that field called text_processing. So they defined that setting in hook_field_info(), and it perpetuates its way across the system.
I don't think default_widget and default_formatter need any more explanation, the docs cover those completely (nothing more to be said about them really, they both accept a string and what that string should be is obvious if you read the argument descriptions).
default_widget and default_formatter, I don't see what the value string should be. The argument description says "provided by the field type module". I could not find where that module is. I only see field and field_ui in the Drupal core modules. I think having documentation more specific as much as possible makes sense. It will definitly help developers to make dev faster and more developers come to use Drupal :)
l() page. I had to follow the documentation of url(). Developer who is not clever may not notice it. Do you think the complete allowed arguments for $options should be listed in l() or linked directly to the $options of url() such as ".../function/url/7#options" for the sake of quick learning?
hook_field_info() is for ultimately. "provided by the field type module" simply means "provided by the module that implements the field type". So for the text widget, the text module is the "field type module". Those docs don't need to be more explicit IMHO - if you read through the whole documentation it'll be easy to spot where your misunderstanding is. Unfortunately reading the docs for one hook/function without any context will never work ;)
l() could be a bit more explicit about where the options array is defined though, yeah