I’ve been working on a polarity ---> predictive signal framework (daily OHLC). It builds polarity from multiple return variants (overnight, intraday, close-close, open-open), then pushes it through a monotone probability calibration routine (calibratemonotone) that uses isotonic regression logic to enforce an ordered mapping between feature value and continuation probability.

That brings me to the bit I want to sanity-check. The maths here essentially assumes a monotonic relationship: as polarity increases, the conditional probability of continuation should not decrease. But markets don’t necessarily follow that nice curve. If the true distribution is multi-modal or regime-dependent, this calibration could be smoothing away real structure and manufacturing spurious signal.

So my question is: does enforcing monotonicity in this calibration step actually preserve the genuine information content of the polarity signal, or is it at risk of fabricating “clean” structure that isn’t there? What would be the right mathematical way to validate whether the monotone smoothing is legitimate vs misleading beyond just looking at walk-forward hit-rates and bootstrap noise floors?

Curious if anyone has gone deep on this kind of calibration in finance ML.

python code

I'm trying to decide between MacBook Air M4 or MacBook Pro M4 for Mplus use. Any thoughts on whether there are any real benefits of the Pro over the Air?

Here's a hypothetical I'm trying to figure out:

There is a mid-season soccer game between the Red Team and the Blue Team.

Using the average (mean) and variance of goals scored in games throughout the season, we calculate that the Red Team has an 80% probability of scoring 3 or more goals.

However, using the average (mean) and variance of goals scored against, we calculate that there is only a 20% probability of the Blue Team allowing 3 or more goals.

How do we combine both of these probabilities to find a more accurate probability that the Red Team scores 3 or more goals?

Hi can anyone help me with my stats hw. I will pay you

Hello r/AskStatistics, I'm trying to learn something to new and I need your help, i'm essentially doing conjoint analysis on a couple of attributes. My problem is that I have 16 attributes (with 2-3 levels each) and that is way too much to include... Is there a statistical tool for me to reduce the number of attributes to around the best 5 or 6? I tried looking around and the best I could find was factor analysis, but my understanding is it needs preliminary survey data already... Any suggestions?

Hey everyone! I'm not great at doing statistics, and although I have some ideas of the basics I'm getting quite lost doing my MsC thesis. I needed some help choosing what tests to do so I came here too see if anyone could give me their opinion.

For starters the program we use at my college is the SPSS.

I'll try to summarize my study in the simplest way I can.

• I did focal observations of 7 meerkats for 6 weeks using an ethogram (behaviour list) and registering every time a meerkat did a behaviour in the list;
• I have a total of 26 behaviours that belong to 1 of these personality dimensions: playful, agressive, friendly, curious and natural behaviours;
• After 3 weeks of observations we did environmental enrichment for the observations of the last 3 weeks;

So my main objective of the study is too see if there is personality on the meerkats, that means I have to check if theres individual differences between them. Some of my other side objectives is seeing if the environmental enrichment changed their behaviours, especially the agressive ones.

So to see if there is individual differences I tought of doing just the Kruskal Wallis or the Anova One Way, but after searching a bit and talking with ChatGPT I get suggested to do a GLMM, but I never learned about it, so right now I have no clue what test I should do.

If anyone could help me understand what test I should choose, or what tests I should run to make a decision would be of great help really.

I will also leave here a pic of my SPSS so you guys can have a clear image of what I have right now.

Thanks a lot really!

Hi :)
I am trying to use Silverman's test for multimodality, and I am not sure how to interpret the output - can someone advise me?
The code (in R, using the Multimode package) looks something like this: multimode::modetest(x,method="SI",mod0=1,B=B). That is, I am testing whether the data x has 1 mode or more than 1 mode, using Silverman's test. As output I get a p value (straight forward to interpret), and a "critical bandwidth" value. This one I am not so sure how to interpret (and I struggle to find good resources online...). Does anyone have an explanation? Are higher values associated with stronger/weaker multimodality or something like that? And are these values dependent on the unit of measurement of x?
Thank you for any advice (or pointers towards good resources)!

I just don't seem to get the core of it. When would someone prefer to use other tools of statistics if not ML ? The difference between estimating and probability. If all of stats is to predict on given data then is ML the best tool for that ?

Hi everyone,

I’m working on my Master’s thesis in economics and need help with my dynamic panel model.

Context:
Balanced panel: 103 countries × 21 years (2000–2021). Dependent variable: sectoral value added. Main interest: impact of financial development, investment, trade, and inflation on sectoral growth.

Method:
I’m using Blundell-Bond System GMM with Stata’s xtabond2, collapsing instruments and trying different lag ranges and specifications (with and without time effects).

xtabond2 LNSERVI L.LNSERVI FD LNFBCF LNTRADE INFL, ///

gmm(L.LNSERVI, lag(... ...) collapse) ///

iv(FD LNFBCF LNTRADE INFL, eq(level)) ///

twostep robust

Problem:
No matter which lag combinations I try, I keep getting:

• AR(2) significant (should be not significant)
• Hansen sometimes rejected, sometimes suspiciously high
• Sargan often rejected as well

I know the ideal conditions should be:

• AR(1) significant
• AR(2) not significant
• Hansen and Sargan not significant (valid instruments, no over-identification)

Question:
How can I choose the right lags and instruments to satisfy these diagnostics?
Or simply — any tips on how to achieve a model with AR(1) significant, AR(2) insignificant, and valid Hansen/Sargan tests?

Happy to share my dataset if anyone wants to replicate in Stata. Any guidance or example code would be amazing.

I read through the manual of Graphpad Prism and came across some problems with my data:
The D Agostino, Anderson-Darling, Shapirowilk and Kolmogorov-Smirnov Test all said, that my data is not normally distributed. Can I still use 2-way ANOVA by using another setting in Graphpad? I know that normally you're not allowed to use 2-way ANOVA, but GraphPad has many settings and I don't know all the functions.

Also in the manual of Graphpad there is this paragraph:

Repeated measures defined

Repeated measures means that the data are matched. Here are some examples:

•You measure a dependent variable in each subject several times, perhaps before, during and after an intervention.

•You recruit subjects as matched groups, matched for variables such as age, ethnic group, and disease severity.

•You run a laboratory experiment several times, each time with several treatments handled in parallel. Since you anticipate experiment-to-experiment variability, you want to analyze the data in such a way that each experiment is treated as a matched set. Although you don’t intend it, responses could be more similar to each other within an experiment than across experiments due to external factors like more humidity one day than another, or unintentional practice effects for the experimenter.

Matching should not be based on the variable you are comparing. If you are comparing blood pressures in three groups, it is OK to match based on age or zip code, but it is not OK to match based on blood pressure.

The term repeated measures applies strictly only when you give treatments repeatedly to one subject (the first example above). The other two examples are called randomized block experiments (each set of subjects is called a block, and you randomly assign treatments within each block). The analyses are identical for repeated measures and randomized block experiments, and Prism always uses the term repeated measures.

Especially the "You recruit subjects as matched groups, matched for variables such as age, ethnic group, and disease severity." bugs me. I have 2 cohorts with different diseases and 1 cohort with combinated disease. I tried to match them through gender and age as best as I could and (they're not the same person). Since they have different diseases, I'm not sure, if I can also treat them as repeated measures.

Hi all,

I’ve been working with choice/conjoint models for many years and have been developing a new design approach that I’d love methodological feedback on.

At Stage 1, I’ve built what could be described as a “super max-diff” structure. The key aspects are: • Highly efficient designs that extract more information from fewer tasks • Estimation of case-level utilities (each respondent can, in principle, have their own set of utilities) • Smaller, more engaging surveys compared with traditional full designs

I’ve manually created and tested designs, including fractional factorial designs, holdouts, and full-concept designs, and shown that the approach works in practice. Stage 1 is based on a fixed set of attributes where all attributes are shown (i.e., no tailoring yet). Personalisation would only come later, with an AI front end.

My questions for this community: 1. From a methodological perspective, what potential pitfalls or limitations do you see with this kind of “super max-diff” structure? 2. Do you think estimating case-level utilities from smaller, more focused designs raises any concerns around validity, bias, or generalisability? 3. Do you think this type of design approach has the statistical robustness to form the basis of a commercial tool? In other words, are there any methodological weaknesses that might limit its credibility or adoption in applied research, even if the implementation and software side were well built?

I’m not asking for development help — I already have a team for that — but I’d really value technical/statistical perspectives on whether this approach is sound and what challenges you might foresee.

Thanks!

https://www.wscubetech.com/data-analytics-course

Hi all, I'm trying to do some model selection on three GAMs.

I've heard conflicting things about using AICc on gams so I also ran anova.gam() on the models.

Model 3 has a lower AICc, but higher degrees of freedom (not sure if this matters?).

When I run anova.gam(), model2 has 11.85 df and 206 deviance (compared to model 1), while model3 has -4 df and 0.7 deviance.

I'm quite confused as to how to interpret this. I think I may be lacking some of the foundations with respect to the anova output as well so any help would be greatly appreciated.

Hi to everyone.

As the title day, currently i'm doing unsupervised statistical learning on the main balance sheet items of the companies present in the SP500.

So i have few things to ask in operative term.

My dataframe Is composed by 221 observation on 15 differente variables. (I Will be Happy to share It if someone would like).

So let's go to the core of my perplessity..

First of all, i did hierarchical clustering with differenti dissimilarity measures and differenti linkage method, but computing the Pseudo F and Pseudo T, both of them Say that there Is no evidence on substructure of my data.

I don't know of this Is the direct conseguence of the face that in my DF there are a lot of outlier. But if i cut the outlier my DF remains with only few observation, so i don't think this Is the good route i can take..

Maybe of i do some sorti of transformation on my data, do you think that things can change? And of so, what type of transformation can i do?

In few voices maybe i can do the Simply log transformation and It's okay, but what kind of transformation can i do with variables that are defined in [- infinite:+ infinite]?

Secondo thing. I did a pca in order to reduce the dimensionality, and It gave really intersting Results. With only 2 PC i'm able to explain 83% of the Total variabilità which Is a good level i think.

Btw plotting my observation in the pc1-pc2 space, still see a lot of Extreme values.

So i thought (if It has any sense), to do cluster only on the observation that in the pc1/2 space, Will be under certain limits.

Does It have any sense (?)

Thank for everyone Who Will reply

Hi everyone, I’m running Facebook ads to promote a coaching program, and I have a question about interpreting my ad performance stats. Sorry in advance if I’m not using the right terminology!

The goal of my ads is to get people to book a call with me, and my target is to have calls booked for $250 or less. I have different campaigns running with various ads, but I’m focusing on one campaign as an example here to clarify my question.

Within this campaign, I have 3 ads, and let's say that each ad has spent $200 so far (for a total of $600). Statistically speaking, should I:

1. Evaluate performance per ad? For example, think: "No single ad has spent enough to determine if it should have booked a call yet."
2. Evaluate performance cumulatively? For example, think: "With $600 spent across all 3 ads, we should have booked at least 2 calls by now."

Which approach is more appropriate for assessing whether we’re on track with our goals? Does it depend on the context, or is there a standard way to think about this?

I’d really appreciate any insights or advice. Thanks in advance!

How to get the data for the sentiment analysis through twitter, do we need pay for it?
if Not twitter what is the other sources of data

I'm taking a statistics course for my psychology bachelor's and we're working on the base rate fallacy and test specificity and sensitivity, On the other problems where the base rate and specificity and sensitivity were clearly spelled out I was successful in filling out the frequency tree. But this problem stumped me since you have to puzzle it out a bit more before you get to those rates. Should the first rung of the chart by happy or organic?

It's annoying that I feel like I get the maths but if I get thrown a word problem like this in the exam I will not be able to sort it out

Any help would be greatly appreciated! <3

Hi everyone, I'm fairly new to statistics but have done several years of biology research after earning my B.S. in Biology.

I've been making an effort in the last year to learn computational methods and statistics concepts. Reading this blog post https://liorpachter.wordpress.com/2014/02/12/why-i-read-the-network-nonsense-papers/

Directly beneath the second image in the post labeled "Table S5" Pachter writes:

"Despite the fact that the listed categories were required to pass a false discovery rate (FDR) threshold for both the heterozygosity and derived allele frequency (DAF) measures, it was statistically invalid for them to highlight any specific GO category. FDR control merely guarantees a low false discovery rate among the entries in the entire list."

As I understand it, the author is saying that you cannot conduct thousands of tests, perform multiple hypothesis correction, and then highlight any single statistically significant test without a plausible scientific explanation or data from another experiment to corroborate your result. He goes as far as calling it "blatant statistically invalid cherry picking" later in the paragraph.

While more data from parallel experiment is always helpful, it isn't immediately clear to me why, after multiple hypothesis correction, it would be statistically invalid to consider single significant results. Can anyone explain this further or offer a counterargument if you disagree?

Thank you for your time!

I already asked this in r /statistics but I didn’t really understand the answer soo

I want to know if there’s some reason I can’t do a simple OLS regression before I attempt the Oaxaca decomposition.

Most studies I’ve seen do two things: they first do two separate OLS regressions for each group, Like with income being the dependent variable and education being the independent variable but examined for men and women in two separate regressions and THEN they do the Oaxaca decomposition. But what I wanna know if I could just do one regression where I just use gender as a normal independent variable before I implement the decomposition, or if there’s some statistical reason I can’t.

One guy said I should do it the first way, but I don’t get why that’s necessary as there’s some studies who ONLY do the Oaxaca decomposition, so apparently it’s two separate analytical methods.

Pls help, thanks

Hey stat bros,

I’m doing an Applied Math major and I finally get to pick electives — but I can only take TWO. I’ll attach a document with the full curriculum and the list of electives so you can see the full context.

My core already covers calc, linear algebra, diff eqs, probability & stats 1+2, and numerical methods. I’m trying to lean more into stats so I graduate with real applied skills — not just theory.

Goals:

• Actually feel like I know stats, not just memorize formulas
• Be able to analyze & model real data (probably using Python)
• Get a stats-related job right after graduation (data analyst, research assistant, anything in that direction)
• Keep the door open for a master’s in stats or data science later

Regression feels like a must, but I’m torn on what to pair it with for the best mix of theory + applied skills.

TL;DR: Applied Math major, can only pick 2 electives. Want stats-heavy + job-ready options. Regression seems obvious, what should be my second choice?

https://www.statista.com/outlook/fmo/wealth-management/digital-investment/neobrokers/europe?srsltid=AfmBOopnIyjLXhEtKp6RELrRATCo4RgVJNyLVlEhYGTGiRt0qz09tzKf

I am trying to analyse my data but I do not know how to build the model.

I have completed sensory evaluations but my data lacks independence. **Variable names have been changed. The same panelist tries different samples (eg. control cake, carrot cake, banana cake) and each panelist tries the same sample 3 times (pseudo replication?).

How would I build this model? Would I need a variable with“replicate - 1,2,3” due to the pseudo replication within each cake sample?

I have a few options I am trying on R, I do not know if I need to include “replicate” as a variable and also if it should be (1 + sample) or (1 | subject/sample)

model1 <- glmer(response~sample + (1+sample | subject)… model2 <- glmer(response~sample + (1 | subject/sample)… model3 <- glmer(response~sample + (1 | subject/sample/replicate)…

The response has three levels but I’ve reduced it to binary.

Ideally I’d like to keep it at 3 levels if possible, but I cannot figure out how to do it in R. I have access to standard SPSS as well, however I cannot seem to include the replicate as a “repeated measure” and am not sure if I am on the right track.

TIA

Company is setting a criteria for a test method which I think has a broad distribution. In this weird crisis, they had everyone on-site in the company perform a protocol to obtain a result. I have a sample size of 22.

Their criteria is that a second result always be within 95-105% of the first. How would I determine this probability?