The ::method as a string is a special feature of the form API.
It's basically a shorthand for [$this, 'method'] and does almost exactly the same. See FormState::prepareCallback
There is one interesting difference, [$this, 'method'] has to serialize the whole form object within the form structure, to then unserialize it to call a method on it. But at that point, on a cached request, it is actually a different object because the form system also created a new form object, as part of initializing it and starting to process the request. ::method will actually call the method on the new form object.
Whenever possible, use the short version, sometimes that's not possible, for example when you are inside a plugin (e.g. a field widget) or other class called by the form, in this case, it is recommend to use a static method.