NetNews Usenet Archive 1992 #30/NN_1992_30.iso/spool/bit/listserv/psycgrad/5021

Text File | 1992-12-14 | 13.7 KB | 256 lines

Comments: Gated by [e-mail]
Path: sparky!uunet!paladin.american.edu!auvm!UTXVM.CC.UTEXAS.EDU!TBN
Message-ID: <[e-mail]>
Newsgroups: bit.listserv.psycgrad
Date:         Sun, 13 Dec 1992 21:18:42 -0800
Sender:       Psychology Graduate Students Discussion Group List
              <[e-mail]>
From:         Tor Neilands <[e-mail]>
Subject: Re: let's hear some opinions
Lines: 244

Monika writes:

>On Fri, 11 Dec 1992 15:27:47 -0800 Tor Neilands said:
>>
>>What does it mean to say "looking at the data the way it is". My
>>understanding of the hypothesis testing framework is that as sample size
>>increases, so does the statistical power to reject the null hypothesis.
>>Thus, as your N increases, you actually will obtain better parameter
>>estimates of population parameters. In this sense, then, if you are
>>interested in estimating the "true" population values of parameters, you
>>wish to have as large a sample as possible--the larger the sample, the more
>>accurate your parameter estimates should be.
>>Practical significance and statistical significance are not the same concepts.
>
>Tor, obviously our appraches to research are very different.  I agree that
>statistical analyses are essential, however, thay are not EXCLUSIVELY
>essential.  If I am testing a behavioural adaptation of a refractory period
>technique, I can just LOOK at the graph in order to determine what this
>refractory period is.  Let is say, that the asymptote of a curve should
>theoretically reach 1.0, however, after testing several animals I realize
>that most of the asymptotes span between 0.4-0.8.  Obviously they are smaller
>than 1.0.  Should I run the analysis to determine whether they are
>statistically different from 1.0?  That is not the point.  If I were to do
>that I would completely miss the boat on what the graph represents.  Such
>values lead us to believe that there might be some inhibitory neural
>mechanisms preventing the curves from appraching 1.0.  Although, this might not
>be the best example, it shows that not all research is about stats and
>significance levels.  I am convinced that if I were to run the analysis I
>would find that 0.4 is statistically different from 1.0, but that information
>would be completely USELESS, because it ISN'T MEANINGFUL!

Why is this information "useless" and "not meaningful"? To you perhaps, but
not to someone else....

For example, what's your question? If your question is whether group A
members differ from group B members in the population from which your
sample was selected, and group A had a sample mean value on your DV of 1.0
and your group B had a sample mean value of 0.4, then you might be
interested in doing an hypothesis test to see if the two groups were
different on mean values of the DV in the underlying population from which
the sample was drawn. In that case, the finding of a statistically
significant difference between group means would be very useful and
meaningful information, in my judgment.

On the other hand, that might not be your question. As you indicate above,
you may wish to plot change graphically, and "eyeball" the results. That
procedure might answer a different question. What the most interesting
questions are, is, of course, up to you, as you are doing the research.

>I believe that by
>blindly relying on statistics instead of looking at what the data may be
>telling us, we often miss the essential.

Please. Anyone who's read any of my other posts to this list over the last
few months will recognize that my approach to both research and statistics
is the antithesis of a reliance on a rigid "hypothesis-testing *only*"
framework. In fact, when I do consulting, I tell users/consultees that one
of the first things they may wish to consider doing is graphically plotting
the means of their groups and eyeballing the results and really "getting
their hands dirty" with the data and attempting to determine what
story(ies) the data tell. However, I also firmly believe that if one is
going to engage in the hypothesis testing game, wherein one takes on a
certain "bet" that there will be a certain chance of making a type 1 error,
and one intends to make certain decisions about the "truth" of an "effect"
based on the obtained p-values from an hypothesis test being above or below
a critical p-value, one should play by the rules of that framework.

So, in summary, yes, I think the hypothesis testing framework isn't for
everyone--in fact, I think it should be used a lot less than it is at
present. I've seen all too many instances of folks running analysis after
analysis in an attempt to find a "significant" finding so that they can
publish something and either get tenure (assistant professors) or get a job
(grad students). To me, this is a most appalling misuse of inferential
statistics, especially given that we don't hear about the 25 analyses that
were conducted before the analysis which yielded p<.05. A lot of this whole
issue for me boils down to the *intent* of what the data analyst is doing.
As one of my stat professors once said, when asked if a certain procedure
was "appropriate", he fired back, "What do you mean by appropriate? You can
*do* anything you want with statistics and numbers. What's important is to
be completely open and honest with your readership about what you did and
why you did it. Then you can defend your actions against criticisms, but at
least everything is stated explicitly for consideration by the readership".

>
>>That's great. I'm very happy for her. Did she have an advisor? Did she have
>>a dissertation committee who provided suggestions to her about what she
>>should do during her preliminary orals? Did she talk to friends and/or
>>colleagues about her research designs? Did she include an introduction in
>>her dissertation write-up? Did that introduction cite previous literature
>>on which her own work is in part based? Did her advisor guide her or work
>>with her in any way?
>
>Tor, she did all this and more.  Her supervisor, which is also my supervisor,
>is excellent in her apprach with students, in a sense that she enables us
>to conduct OUR, INDEPENDENT RESEARCH (after all isn't getting a Ph.D about
>thinking for yourself and solving problems with your own experience,
>knowledge and intuition?) with conjunction of occasional, sometimes weekly,
>sometimes biweekly meetings.  Her office is beside ours, and we can always pop
>in for a question, however, we know more about our research than she does,
>and since she has 4 of us, we need to bring her up on the newest stuff,
>because she simply forgets what each one of us is doing.  My collegue
>did not consult her commitee before she defended, except for submitting
>her proposal a year before (when she had already most of her data collected).
>She often discussed with us (especially with me, because our research interests
>overlap) her data and the interpretations of it.  Of course, she
>included an introduction and summary of litterature on which her research
>was based!  How could she call it a PH.D disseration if she did not?!
>Your question is rather surprising, but maybe I missunderstood your
>intentions.  Our supervisor did not guide her work.  They both designed
>an experiment, but my collegue was responsible for comming up with
>practical and theortical solutions when things did not work out (everyone
>reaches a "dead point" in their research at one time or another).
>The greatest advantage of working with our supervisor is that she is a
>perfectionist when it comes to writing, the final written products
>comming our of our lab are always very well received by the community.

My questions here were designed to illustrate that all of us are dependent
on the past work of others to guide our thinking when it comes to a
dissertation proposal write-up (hence the need for an introduction
section), the insights of friends and advisors/supervisors, etc. So, please
excuse me for being dense, but I don't see how your friend could do an
"independent dissertation" when it's obvious she's had input and
contributions from other colleagues which have influenced the final
dissertation product?

And, no, I don't think obtaining a PH.D. is *necessarily* about

>thinking for yourself and solving problems with your own experience,
>knowledge and intuition?)
>
I think that can be part of what a PH.D. is about. It can also be about
learning the value of teamwork, cooperation, and gaining the insights of
others in order to develop a higher quality dissertation. I'm not the only
one who believes in the cooperative model of higher education, either: one
of my committee members commented at my prelim orals that he frequently
sits on committees in another department where they do research related to
his speciality, and their students turn out less high quality final
disserations because, in his judgment, they do not require their students
to go through prelim orals---and the students' final dissertations suffer
in quality as a result of the lack of input of the entire committee.

Two of my committee members directly commented to me that they thought I
had come up with a higher quality dissertation proposal because I had gone
to the trouble of iterating it through *all* committee members for comments
and criticisms several times before my prelims took place.

>>I hope to receive my doctorate in may. When I receive it, I'll be happy and
>>proud that while I did a lot of work on this project and put a tremendous
>>amount of effort into it for the past three years, I haven't gone at it
>>alone--I've learned the value of teamwork and give and take with my advisor
>>and my committee members; through gleaning suggestions from them my
>>proposal became *much* stronger as a result of their input and criticism. I
>>also learned the value of educating and working with enthusiastic,
>>motivated undergraduates who very much wanted to take advantage of the
>>opportunity to do research with me and my advisor. In fact, I had so many
>>applicants for the position that I had to turn away more applicants than I
>>could accept. And, on a purely pragmatic level, many grad schools require
>>that their undergrad applicants have some type of research experience under
>>their belts. By providing the undergrads with an opportunity to work with
>>us, they not only get that necessary pragmatic credential, they also get to
>>"try out" psychological research and see if they like it--and that's an
>>important service we can provide to the undergraduate population of
>>psychology majors.
>
>Tor, Best of luck, I hope everything goes well for you and in May you will
>be able to call yourself Dr.Tor...!

Thanks!

>I also learned value of team work, since I share office and lab space with
>4 other lab mates, however, I would never trust them with my own research
>because I am the one who knows best how it ticks.

I'm very sorry to hear that. I would imagine that this belief would make it
very difficult to accept criticisms for journal editors, colleagues, etc,
because, after all, they know your research less well than you and
therefore what they say, while having some truth or validity to it, will
never be as "correct" as your own beliefs on the subject.

>I also believe that it is important for the undergrads to
>gain some real research experience, I would just prefer that they do it
>with their own data , in a form of a research assistantship, honours thesis or
>the like...(at least that is what worked for me).

I'm glad to hear that that system worked for you. Here we have more
undergraduates who want very desperately to learn more about research and
to be exposed to it, more than can possibly accomodated by the honors
course here, or by other research courses where they run their own subjects
on their own projects. Furthermore, in all of these courses, the projects
are as much the professors' and TAs' projects as they are the students'
because the professors' and TAs' "guide" the undergrads in focusing their
research ideas, defining questions of interest, what literature to search,
etc.
>
>
>                                 _
>    ) (    Monika Trzcinska    _- o   ) (   [e-mail]
>   )   (----------------------- o o  )   (----------------------------
>    ) (                        -_ o   ) (
>                                 -
>   "The most important attitude that can be formed is that of a desire
>    to go on learning".
>                                                  Dewey (1938)

In sum, I think the idea of doing "truly independent" research is a
fallacy. No one is an island unto him or herself. I agree that one should
exhibit the most flexibility possible in gaining information about research
topics of interest and in exploring data with different analytic
techniques, as long as one plays by the rules when one engages in
inferential statistical tests designed to test specific hypotheses and,
above all, being sure to inform one's readership/audience what one did
step-by-step in the research methods and data analytic stages of the
research process.

cheers,

Tor


Tor Neilands
Systems Analyst
Statistical Services Group
Computation Center
University of Texas at Austin
Austin, TX  78712

Phone: [phone], Ext. 263

Internet:  [e-mail]
___________________________________________________

"To thy own self be true".             --- Walt Whitman

"Two roads diverged in a yellow wood and I, I took
 the road less traveled by, and that has made all the
 difference".                                 --- Robert Frost
___________________________________________________

Disclaimer:  All of the views expressed above are solely
those of the author. Any relationship between my own
views and those of the University of Texas and/or The
Computation Center at the University of Texas are
due merely to regression to the mean.
____________________________________________________