Scene Segmentation with SynF

[amyvanee]

#39 My issue is about adjusting Proglearn so we can do scene segmentation after flattening the images.

Reproducing code example:

def load_images(flatten_imgs):
    if flatten_imgs:
        X = np.array([cv2.imread(imgpath).flatten() for imgpath in imgpaths])
        Y = np.array([cv2.imread(lblpath).flatten() for lblpath in lblpaths])
    else:
        X = np.array([cv2.imread(imgpath) for imgpath in imgpaths])
        Y = np.array([cv2.imread(lblpath) for lblpath in lblpaths])
    return X,Y

flatten_imgs = True
X, Y = load_images(flatten_imgs)

from random_class_functions import Odif_experiment

slot_num = int(5000 / num_points_per_task)
slot_fold = range(slot_num)
shift_fold = range(1, shift_num + 1, 1)

# run the Odif model
n_trees = [tree_num]
iterable = product(n_trees, shift_fold, slot_fold)
df_results = Parallel(n_jobs=-1, verbose=0)(
    delayed(Odif_experiment)(
        X, Y, ntree, shift, slot, num_points_per_task, acorn=12345
    )
    for ntree, shift, slot in iterable
)

Error message

AttributeError                            Traceback (most recent call last)
<ipython-input-8-097c14639e16> in <module>
     10 n_trees = [tree_num]
     11 iterable = product(n_trees, shift_fold, slot_fold)
---> 12 df_results = Parallel(n_jobs=-1, verbose=0)(
     13     delayed(Odif_experiment)(
     14         X, Y, ntree, shift, slot, num_points_per_task, acorn=12345

~\AppData\Local\Programs\Python\Python39\lib\site-packages\joblib\parallel.py in __call__(self, iterable)
   1052 
   1053             with self._backend.retrieval_context():
-> 1054                 self.retrieve()
   1055             # Make sure that we get a last message telling us we are done
   1056             elapsed_time = time.time() - self._start_time

~\AppData\Local\Programs\Python\Python39\lib\site-packages\joblib\parallel.py in retrieve(self)
    931             try:
    932                 if getattr(self._backend, 'supports_timeout', False):
--> 933                     self._output.extend(job.get(timeout=self.timeout))
    934                 else:
    935                     self._output.extend(job.get())

~\AppData\Local\Programs\Python\Python39\lib\site-packages\joblib\_parallel_backends.py in wrap_future_result(future, timeout)
    540         AsyncResults.get from multiprocessing."""
    541         try:
--> 542             return future.result(timeout=timeout)
    543         except CfTimeoutError as e:
    544             raise TimeoutError from e

~\AppData\Local\Programs\Python\Python39\lib\concurrent\futures\_base.py in result(self, timeout)
    443                     raise CancelledError()
    444                 elif self._state == FINISHED:
--> 445                     return self.__get_result()
    446                 else:
    447                     raise TimeoutError()

~\AppData\Local\Programs\Python\Python39\lib\concurrent\futures\_base.py in __get_result(self)
    388         if self._exception:
    389             try:
--> 390                 raise self._exception
    391             finally:
    392                 # Break a reference cycle with the exception in self._exception

AttributeError: 'bool' object has no attribute 'any'

Version information

• OS: [e.g. macOS] Windows
• Python Version [e.g. 3.7.3]: Python 3.7.3
• Package Version [e.g. 0.0.1]: Proglearn 0.0.5

[jdey4]

@amyvanee did you try fitting proglearn for one task only? I would not try it with other people's code. Please try to train it on only one task using add_task first. Let's not use parallel for now.

[PSSF23]

@amyvanee Which version of proglearn are you using?

[amyvanee]

@PSSF23 Sorry about that, I am using Proglearn 0.0.5

[amyvanee]

@jdey4 I tried going to the basics more and initializing a LifelongClassification Forest, but I got an error.

%  Declare the progressive learner model (L2F)
learner = LifelongClassificationForest()

% add the task
learner.add_task(
                flat_x,
                flat_y,
                task_id=0
                )

And the output,

TypeError                                 Traceback (most recent call last)
TypeError: only size-1 arrays can be converted to Python scalars

The above exception was the direct cause of the following exception:

ValueError                                Traceback (most recent call last)
<ipython-input-18-74e8ef13550e> in <module>
      3 
      4 # add the task
----> 5 learner.add_task(
      6                 flat_x,
      7                 flat_y,

~\AppData\Local\Programs\Python\Python39\lib\site-packages\proglearn-0.0.5-py3.9.egg\proglearn\forest.py in add_task(self, X, y, task_id, n_estimators, tree_construction_proportion, kappa, max_depth)
    122             max_depth = self.default_max_depth
    123 
--> 124         X, y = check_X_y(X, y)
    125         return self.pl_.add_task(
    126             X,

~\AppData\Local\Programs\Python\Python39\lib\site-packages\sklearn\utils\validation.py in check_X_y(X, y, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, multi_output, ensure_min_samples, ensure_min_features, y_numeric, estimator)
    954         raise ValueError("y cannot be None")
    955 
--> 956     X = check_array(
    957         X,
    958         accept_sparse=accept_sparse,

~\AppData\Local\Programs\Python\Python39\lib\site-packages\sklearn\utils\validation.py in check_array(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, estimator)
    736                     array = array.astype(dtype, casting="unsafe", copy=False)
    737                 else:
--> 738                     array = np.asarray(array, order=order, dtype=dtype)
    739             except ComplexWarning as complex_warning:
    740                 raise ValueError(

~\AppData\Local\Programs\Python\Python39\lib\site-packages\numpy\core\_asarray.py in asarray(a, dtype, order)
     81 
     82     """
---> 83     return array(a, dtype, copy=False, order=order)
     84 
     85 

ValueError: setting an array element with a sequence.

I double checked, and below are shapes,

• flat_x.shape = (64,)
• flat_y.shape = (64,)
• flat_x[0].shape = (10890000,)
• flat_y[0].shape = (9000000,)
• data_x[0].shape = (2200, 1650, 3)
• data_y[0].shape = (2000, 1500, 3)

If I try putting in any of the other data, I get an error that the input should be 2D.

Also, it is weird that the shape of data_y[0] is not the same as for data_x[0]?

Thank you for your help!

[PSSF23]

@amyvanee always use the latest version, which is 0.0.6 now. And what are data_x & flat_x & ... ? We have no idea what they are, but the y seems too large. What are the labels of these images?

[amyvanee]

@PSSF23

• Thank you! I tried updating to Proglearn 0.0.6, using
  pip install proglearn --upgrade
  but I got an error.
  error: Could not find module 'hdf5.dll' (or one of its dependencies). Try using the full path with constructor syntax. Loading library to get version: hdf5.dll

• The images are from the ADE20K consistency set

• There are 64 images, each 2D with RGB so the image arrays are 3D. X is the original image, and Y is the image where each pixel is annotated by the object it is. (I am not sure why, but the array dimensions do not match up between X and Y)

• data_x and data_y are the original images and the annotated image. flat_x and flat_y are the flattened version of these images, since we thought flattening them to be 1D would help with insertion into proglearn.

Thank you for your help!

[PSSF23]

• The error is unrelated to proglearn. I have never seen it before.
• Your fitting process doesn't make sense. Based on your descriptions, each y image is different so there's no valid classification.

[PSSF23]

BTW are you working on #39 ? Why didn't you comment on it?

[amyvanee]

@PSSF23

• Thank you, I will try seeing if I can fix it another way!
• Yes that was my concern since I know proglearn takes in the true labels as one label per image (so a 2D image where the output is, for example, "dog") -- would I need to make modifications to the proglearn code to do this? I know scikit actually has this feature with random forests, and it seems to work. I still need to quantify how accurate it is.
• Yes, I am working on issue implement scene segmentation #39! Sorry I should have commented there.

[PSSF23]

@amyvanee No problem. As the example in your link, sklearn RF must rely on skimage to process the images. You can definitely follow it and try it on proglearn. Odif (LifelongClassificationForest) would replace the RandomForestClassifier in that example.

Comment on #39 so it could be assigned to you. Are there other students sharing the issue?

[amyvanee]

@PSSF23 Thank you, I will try that! Yes, I am working with Narayani Wagle (@nhwagle) and Kevin Rao (@KhelmholtzR) We made a separate GitHub repo with some of our progress

[PSSF23]

@amyvanee @nhwagle @KhelmholtzR you should all comment on #39 and create your independent sub-issue here. So we can keep track of what you are working on.