percepnet implemented using Keras, still need to be optimized and tuned.

Hi cookcodes,

Did you use the self-generated rnn_data.h when running this project?

Recently I tried to build and train keras model myself, and when I dumped model using dump_rnn_float.py, I found that the rnn_data.h generated by command python dump_rnn_float.py weights.hdf5 rnn_data.c rnn_data.h orig is quite different from the rnn_data.h in /src. And apparently the one in /src is much like the correct one.

So I was wondering what's the purpose/use of this self-generated rnn_data.h? Did you use this file in this project? Thanks!

Thank you for very useful source code. I'm a noob web developer, but I need to deploy your PercepNet model to my web browser (client-side). Which are the feasible ways to do that?

Hi, thanks for your work, I try to use the 16k sampling rate data (DNS) to training. I modify the high frequency in erbband.c from 20000 to 8000 and change the frame size from 480 to 160. I can success to compile the code using compile.sh, and run this command. ./denoise_training ~/speech_folder ~/noise_folder 100 training.f32

But encounter the Nan problem when create the dataset, the log is shown below,

alpha[30] is NAN, a= nan, r[30] is NANalpha[31] is NAN, a= nan, r[31] is NANalpha[32] is NAN, a= nan, r[32] is NANalpha[33] is NAN, a= nan, r[33] is NANEphaty[30] is Nan detected. r[30] is Nan detected Ephaty[31] is Nan detected

Is there anything I miss? Thank you

computer_band_energy() in Percepnet is like this:

`ERBBand *erb_band = new ERBBand(WINDOW_SIZE, NB_BANDS-2, 0, 20000);

void compute_band_energy(float bandE, const kiss_fft_cpx X) { int i; float sum[NB_BANDS] = {0}; for (i=0;i<NB_BANDS;i++) { int j; int band_size; band_size = (erb_band->nfftborder[i+1]-erb_band->nfftborder[i]); for (j=0;j<band_size;j++) { float tmp; float frac = (float)j/band_size; tmp = SQUARE(X[(erb_band->nfftborder[i]) + j].r); tmp += SQUARE(X[(erb_band->nfftborder[i]) + j].i); sum[i] += (1-frac)tmp; sum[i+1] += fractmp; } / //ERBBand cosfilter is not working in interp_gain int low_nfft_idx = erb_band->filters[i].first.first; int high_nfft_idx = erb_band->filters[i].first.second; for(j=low_nfft_idx; j<high_nfft_idx; j++){ float tmp; tmp = SQUARE(X[j].r); tmp += SQUARE(X[j].i); sum[i] += tmp*erb_band->filters[i].second[j-low_nfft_idx]; } **/ } sum[0] *= 2; sum[NB_BANDS-1] *= 2; for (i=0;i<NB_BANDS;i++) { bandE[i] = sum[i]; } }`

and yours like this:

` void compute_band_energy(float *bandE, const kiss_fft_cpx X) { int i; float sum[NB_BANDS] = {0}; for (i=0;i<NB_BANDS;i++) { int j; int low_nfft_idx = erbweights[i].begin; int high_nfft_idx = erbweights[i].end; for(j=low_nfft_idx; j<high_nfft_idx; j++){ float tmp; tmp = SQUARE(X[j].r); tmp += SQUARE(X[j].i); sum[i] += tmperbweights[i].weights[j-low_nfft_idx]; } }

for (i=0;i<NB_BANDS;i++) { bandE[i] = sum[i]; } } `

My question is: the comment of compute_band_energy() in Percepnet write that ERBBand cosfilter is not working in interp_gain, is it correct? if it's correct why do you still use cos filter? thanks very much

Hi cookcodes! Thanks for sharing your code with us! In the PercepNet paper, there is a scale bands module after pitch filtering which uses the pitch enhanced signal z and the postfilter output Xb_hat. However, there has few details about this module in paper. I want to know what your idea is about the scale bands module. By the way, could you please share the msse value of gb and rb after convergence in your experiments?

Hi, thanks for your work, I encounter two problems:

1. I set count as 10000000, and use original 48K speech and noise as dataset. the output looks like this:

x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:1
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:2
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:3
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:4
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:5
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:6
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:7
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:8
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:9
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:10
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:11
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:12
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:13
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:14
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:15
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:16
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:17
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:18
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:19
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:20
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:21
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:22
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:23
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:24
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:25
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:26
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:27
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:28
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:29
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:30
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:31
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:32
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:33
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:34
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:35
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:36
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:37
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:38
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:39
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:40
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:41
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:42
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:43
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:44
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:45
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:46
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:47
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:48
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:49
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:50
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:51
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:52
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:53
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:54
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:55
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:56
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:57
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:58
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:59
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:60
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:61
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:62
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:63
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:64
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:65
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:66
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:67
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:68
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:69
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:70
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:71
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:72
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:73
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:74
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:75
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:76
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:77
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:78
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:79
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:80
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:81
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:82
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:83
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:84
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:85
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:86
x_lp[0] is Nan detected after pitch_downsample, so not filtered. total count:87
Segmentation fault (core dumped)

I retry some times, segmentation fault occurs every time after total count achieve 87. I have no idea about that. I set count as 1000000, it works as normal, so I use this config to train model and encounter the second problem.

2. when I run rnn_train.py (tensorflow-gpu 2.5.0), raise error like this:

Traceback (most recent call last):
  File "rnn_train.py", line 206, in <module>
    callbacks=[checkpoint_cb],
  File "/home/edev/.local/lib/python3.6/site-packages/keras/engine/training.py", line 1122, in fit
    steps_per_execution=self._steps_per_execution)
  File "/home/edev/.local/lib/python3.6/site-packages/keras/engine/data_adapter.py", line 1348, in get_data_handler
    return DataHandler(*args, **kwargs)
  File "/home/edev/.local/lib/python3.6/site-packages/keras/engine/data_adapter.py", line 1136, in __init__
    adapter_cls = select_data_adapter(x, y)
  File "/home/edev/.local/lib/python3.6/site-packages/keras/engine/data_adapter.py", line 978, in select_data_adapter
    _type_name(x), _type_name(y)))
ValueError: Failed to find data adapter that can handle input: <class '__main__.CustomDataGen'>, <class 'NoneType'>

can you please tell me your TensorFlow version?

update: I changed some code and fixed the second problem, however, only CPU can be used to train, when I use GPU, raising error: Unknown: CUDNN_STATUS_BAD_PARAM

can you please tell me if you can use GPU to train?

1. As you mention that some performance is worse than rnnoise, can you post some examples ?
2. The DNS-chanllenge dataset is the master branch's fullband data, interspeech2020 or interspeech2021 ? There is some evaluation?