Default behavior of consumer errors in the pipeline

[bogdandrutu]

As of today in the collector main receiver OTLP receiver, errors are by default treated as retryable unless otherwise specified by the source by using the consumererror.NewPermanent. This requires components to accordingly mark every error permanent or not.

Based on the OTLP protocol though, only a very small subset of cases must be retryable, see https://github.com/open-telemetry/opentelemetry-proto/blob/main/docs/specification.md#failures-1.

In contrib right now, looking at some of the most used processor attributes, resource, resourcedetector, and transform:

1. They are all returning errors in some scenarios like failing some filtering conditions, etc.
2. None of these components are returning permanent errors, even though I am highly confident that these errors are deterministic and a retry will not change the behavior.

Proposal:
Since, I've seen this problem in practice, I would like to propose that we change the behavior of the consumer pipeline errors and treat all errors as permanent by default unless specified otherwise.

[dmitryax]

I agree. I remember we discussed this and decided to make the retriable errors opt-in as part of the effort to revisit error propagation that needs to be revamped: #7047, #11085