Designing a new Parser API for Adapters

[tenthe]

Design of a new parser API for adapters

In this discussion, we will explore how to design the new parser API.
Adapters that can handle different formats need a parser (e.g. FileReplay, MQTT, Kafka, ...)
In the connect refactoring branch I have implemented a first version which has the following disadvantages:

Disadvantages of the current API:

• The adapter has no control over the actual events
• If it is necessary to customize events inside the adapter, the developer has to change the adapter's preprocessing pipeline

Current API Design

The current design delegates the parsing and publishing of events to the parser.

Parser Interface

public interface Parser {
...
    void parse(InputStream inputStream, IEventCollector collector) throws ParseException;
...
}

Adapter

public class ActualAdapter implements AdapterInterface {
...
    @Override
    public void onAdapterStarted(IAdapterParameterExtractor extractor, IEventCollector collector,
            IAdapterRuntimeContext adapterRuntimeContext) throws AdapterException {
        InputStream inputStream = // create inputStream from data
        extractor.selectedParser().parse(inputStream, collector); 
    }
...
}

Design 1: Iterator Pattern

One possible improvement is to use the Iterator pattern to give the adapter more control over the actual events.

Parser Interface

public interface Parser {
...
    void init(InputStream inputStream);
    boolean hasNext();
    Map<String, Object> next();
...
}

Adapter

public class ActualAdapter implements AdapterInterface {
    @Override
    public void onAdapterStarted(IAdapterParameterExtractor extractor, IEventCollector collector,
            IAdapterRuntimeContext adapterRuntimeContext) throws AdapterException {
...
        InputStream inputStream = // create inputStream from data
        var parser = extractor.selectedParser();
        parser.init(inputStream);
        while(parser.hasNext()) {
            try {
                var event = parser.next();
                collector.collect(event);
            } catch (ParseException e) {
                // LOG exception
            }
        }
    }
...
}

Design 2: Callback

Another possible improvement is to use a callback to publish the events.

Parser Interface

public interface Parser {
...
    void parse(InputStream inputStream, EventHandler eventHandler) throws ParseException;
...
}

public interface EventHandler {
    void handle(Map<String, Object> event);
}

Adapter

public class ActualAdapter implements AdapterInterface {
...
    @Override
    public void onAdapterStarted(IAdapterParameterExtractor extractor, IEventCollector collector,
            IAdapterRuntimeContext adapterRuntimeContext) throws AdapterException {
        InputStream inputStream = // create inputStream from data
        extractor.selectedParser().parse(inputStream, (Map<String, Object> event) -> {
            collector.collect(event);
        });
    }
...
}

Question

• Is it possible for Design 2 to catch ParseExceptions?
• Do you have an opinion about these designs? Which one do you like best?

[dominikriemer]

Hi @tenthe, I prefer Design 2, as it seems to be easily understandable by developers of adapters.
In the current implementation, it might be difficult to understand that two different methods are used to send events to the collector (parse and collect).
In Design 2, responsibility between parse and collect is clear and it seems to be less code than option 1.

[tenthe]

For option 2, we would need two interfaces that publish events, one for the event collector that sends the events to the adapter's preprocessing pipeline, and one for the new event handler.

public interface IEventHandler {
  void handle(Map<String, Object> event) throws ParseException;
}

public interface IEventCollector {
  void collect(Map<String, Object> event);
}

The interfaces have a different meaning, but they look the same. Do you think that can be confusing?

[bossenti]

I have much less understanding of the affected part of the codebase than you, @tenthe and @dominikriemer, but from my perspective this is indeed confusing. I would say (but I have little feel for good Java syntax) that option 2 is/looks more elegant.
However, option 1 is - at least from my point of view - easier to understand and more lightweight. But since I don't really have a good idea of Java code, my opinion is not really strong here and I'm happy to hear your arguments :)

[tenthe]

Thanks for your feedback @bossenti.
I have started to implement option 2. Once the branch is ready, you can take a look at it and we can discuss if it is confusing to use or if we need to adjust anything.