[linrad] RE: noise blanking/weak signal comparisons

[W6WO <ron-skelton@xxxxxxxxxxx>]

Thank you for the clarifications Leif.

I realize that W3SZ's files would not be enough upon which to make 
any judgements but I was struck by how poor the weak CW copy was and 
yet how good was the SSB copy.

It would be very informative to use recordings of both strong and 
weak HF beacons under quiet and noisy conditions to obtain better 
performance benchmarks over time.

With regard to Wavelet techniques my interest was peaked a few years 
ago by the following.
Announcing a paper ...
To appear in Signal Processing, August 2001 Vol 81/9, pp 1909-1926

Title:   Wavelet Packets and De-noising Based on Higher-Order-Statistics
          for transient Detection

Authors: Philippe Ravier and Pierre-Olivier Amblard

ABSTRACT:

In this paper, we present a detector of transient acoustic signals
that combines two powerful detection tools: A local wavelet
analysis and higher-order statistical properties of the signals.
Using both techniques makes detection possible in low
signal-to-noise ratio conditions, when other means of detection
are no longer sufficient. The proposed algorithm uses the adapted
wavelet packet transform. It leads to a partition of the signal
which is `optimal' according to a criterion that tests the
Gaussian nature of the frequency bands. To get a time dependent
detection curve, we perform a de-noising procedure on the wavelet
coefficients: The Gaussian coefficients are set to zero. We then
apply a classical method of detection on the time reconstructed
denoised signal.
We study the performance of the detector in terms of experimental
ROC curves. We show that the detector performs better than
decompositions using other classical splitting criteria. In a last
part, we present an application of the algorithm on real flow
recordings of nuclear plant pipings. The detector indicates the
presence of a missing body in the piping at some instants not seen
with a classical energy detector.

KEY-WORDS:

transient detection, wavelet packets, adapted segmentation,
de-noising, higher-order statistics, ROC performance curves.

I subsequently delved a bit more into the very active wavelet arena 
and became excited that a lot of work has been done to remove noise 
from signals in other domains such as astronomy and medical. I did 
some experiments using a borrowed wavelet toolkit. Such toolkits are 
part of expensive products eg. Matlab and IDL.  I did get some 
encouraging results but was soon out of my depth. In addition I lost 
the free use of the toolkit and CPU horsepower so work came to an 
end. I have not seen any references to using such techniques in the 
ham world.

Incidentally the weak signal set up I have is a TenTec HF transceiver 
with 2 stages of 250 Hz IF Xtal filters and internal DSP noise 
reduction followed by an outboard audio DSP filter designed by 
VK3GJZ. This uses the Alesis DSP chip and a 100 Hz filter designed 
for CW signals. This is my standard of comparison for weak signals, I 
do not have satisfactory noise blanking.

I am looking forward to getting Linrad up but LINUX is posing a steep 
learning ramp for me.



>Hi Ron,
>
>>  Greetings all. I am interested in weak signal HF CW and have been
>>  drawn to investigate Linrad because of Lief's reputation and my
>>  desire to try to be just behind the bleeding edge. I haven't got
>>  Linrad running yet so I was especially interested in the WAVE files
>>  W3SZ provided. I listened to them carefully.
>Please note that Linrad still has only weak signal CW mode, the
>SSB mode is actually still the weak cw mode but you can put an
>alternative set of parameters that fit SSB bandwidth.
>
>As a consequence the blanker does not work well on strong signals
>unless one makes adjustments to the blanker levels which in turn will
>not make the blanker optimal for weak signals. You can hear this
>malfunctioning as a strong distortion on the loud SSB signal while
>the weaker station is not distorted.
>
>The mechanism is that the frequency range that you selected, the
>frequency range of a strong SSB signal in Rogers recording, will
>be routed together with the noise floor and the weak signals through
>the noise blanker. In weak CW mode the blanker "knows" that the
>total power of the desired signal is much smaller than the total
>power of the noise floor, something that is incorrect in this case.
>
>>  I would like to hear any preliminary conclusions about the
>>  performance of the K2, Linrad and SDR -1000 on both CW and SSB
>>
>>  With regard to noise reduction am I right in my understanding that
>>  the noise cancellation/blanking in Linrad and SDR-1000 is focused on
>>  impulse noise and not on reduction of background noise associated
>>  with lowband HF propagation ?
>I can not say anything about SDR-1000 but the presently implemented
>routines of Linrad are for wideband impulse noise only. The Linrad
>blanker will be extremely efficient for powerline noise, electrical
>fences, car ignition noise and most of your local QRN.
>
>On HF bands there is noise from distant thunderstorms and those
>pulses are distorted by multipath propagation in such a way
>that Linrad can not resolve the individual pulses and therefore
>the pulse removal does not work. Lirad has to rely on conventional
>blanking and is very efficient for short "noise bursts". Linad should
>be far better than K2 or any other receiver in case there are
>strong signals near the desired signal for short "noise bursts".
>
>>  Has anyone in this community investigated de-noising functions
>>  provided by Wavelet techniques ?
>I do not know what "Wavelet techniques" means, what the fundamental
>theory is. To do something that is more clever than to attenuate
>the signal, possibly down to zero (blank it out) when the S/N ratio
>is lower due to an increased noise level one has to have some
>information about the noise source. I have very little experience
>with HF signals, but I have not been able to find any information
>in the HF "noise bursts" so I do not think anything better is possible.
>
>Linrad does not yet have the procedure to remove longer "noise bursts"
>because I do not have suitable test signals. I do not have any
>reasonable HF antennas and I do not know what will be typical in
>"real life" so I am waiting for good recordings of difficult
>situations on HF bands. There is a line "Reserved for blanker"
>in the baseband graph. This is the area for the control functions
>that will be needed for the procedure that will take care of
>longer "noise bursts". I guess one could use Wavelet techniques,
>but since I do not know what it is I may use some simple procedure
>in the time domain. CPU load is not any problem because the baseband
>data rate is low.
>
>73
>
>Leif  /  SM5BSZ

-- 
73
Ron
W6WO

LINRADDARNIL