What is the proper way to return the results of custom module's form submission?

Question

After spending hours and hours online researching, I'm more confused than ever about something that seems like it should be a very straight-forward task - Using my module's custom form, How do I display the results of a form submission that queries the db?

Here on d.se, Clive and Kiamlaluno provide 2 approaches. Clive suggests using a multistep form, and Kiamlaluno suggests making values globally accessible by storing them in the $_SESSION variable.

Clive's approach doesn't make much sense to me. I'm not trying to print the value of a form field (after checking if it has been submitted), I'm trying to print a list of items retrieved from a database.

Kiamlaluno's approach seems a bit more workable. Indeed I could query a db, store the results in the session, and return those as the return value of a page callback on another page. Ultimately, however, this seems awfully convoluted. First, if this were indeed a best practices way, then why don't more tutorials discuss details about it? What logic would be used to set and unset the variable? PHP is horrible at handling large arrays - what if I have a large result set? How do I handle pagination if I want to divide up the result set across pages? Also, isn't the session variable available only to logged in users?

Looking at Views, Views exposed filters seem to use URL query strings. Is this the correct way? After a form submits redirect to another page, append the query string to that page, pass those parameters 'page arguments', and then query and return data in the page callback based on those arguments?

On Drupal.org there is a thread that has almost 4 years of activity from people confused about this very topic. This comment says that you have to create a $form_state['storage'] variable and "then print it in your form definition." What in the world does that mean?!?!?!?

So, I'll pose my question again. I want to query a database using a form in my custom module, and I want to return a list of results (preferably paginated).

How can I accomplish this? I'm looking for an extremely thorough answer that I can play with in practice.

Answer 1

As JimaJamma noted in the comments the search module contains an excellent example of one way to do this. The search_view() function is where the magic happens, this is a custom version of that logic which should help explain the process:

function MYMODULE_menu() {
  $items['query'] = array(
    'title' => 'Query',
    'access arguments' => array('access content'),
    'page callback' => 'MYMODULE_results_page',
  );

  return $items;
}

function MYMODULE_results_page($parameter = NULL) {
  // Even if 'page arguments' isn't defined in a hook_menu() item, Drupal breaks
  // up the URL and passes arguments through to the callback function implicitly.
  // So if the page requested is 'query/123', out item defined at
  // 'query' will be used, and '123' will be passed as the first argument, in
  // this case into the $parameter var.
  //
  // If the parameter is empty, we need to check the GET/POST request to see if
  // the form has been submitted with a new search parameter.
  if (empty($parameter)) {
    if (!empty($_REQUEST['parameter'])) {
      // Build up a new path based on the submitted parameter.
      $path = 'query/' . $_REQUEST['parameter'];

      // And re-direct there. The request will come straight into this function
      // again, but this condition won't fire.
      drupal_goto($path);
    }
  }

  // Now we're in a position to query for results if there's a non-empty
  // parameter. Otherwise we just show the form (like the search page we'll
  // show the form unconditionally).
  $results_build = array('#markup' => '');
  foreach (MYMODULE_get_results_by_parameter($parameter) as $result) {
    $results_build[] = MYMODULE_build_result_render_array($result);
  }

  $build = array(
    'form' => drupal_get_form('MYMODULE_query_form'),
    'results' => $results_build,
  );

  return $build;
}

function MYMODULE_query_form($form, &$form_state) {
  // Make we're submitting to the right path.
  $form['#action'] = 'query';

  $form['parameter'] = array(
    '#type' => 'textfield',
    '#title' => t('Enter query here'),
  );

  $form['submit'] = array(
    '#type' => 'submit',
    '#value' => t('Submit'),
  );

  return $form;
}

I haven't tested that or anything so excuse any small bugs. If there are any bugs hopefully the comments, and the comments in the search_view() function, will do enough to help explain what should be happening :)