< Prev | Home | Model | Random | Next >

Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 6044 in Layer 10

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')



Text #0

Max Range: 0.7097. Min Range: -0.7097

Max Act: 0.6940. Min Act: -0.0000

Data Index: 890183 (C4 (Web Text))

Max Activating Token Index: 958

Click toggle to see full text

Truncated

Full Text #0


Text #1

Max Range: 0.7097. Min Range: -0.7097

Max Act: 0.7097. Min Act: -0.0000

Data Index: 127046 (C4 (Web Text))

Max Activating Token Index: 751

Click toggle to see full text

Truncated

Full Text #1


Text #2

Max Range: 0.7097. Min Range: -0.7097

Max Act: 0.6250. Min Act: -0.0000

Data Index: 200496 (C4 (Web Text))

Max Activating Token Index: 942

Click toggle to see full text

Truncated

Full Text #2


Text #3

Max Range: 0.7097. Min Range: -0.7097

Max Act: 0.5977. Min Act: -0.0000

Data Index: 347676 (C4 (Web Text))

Max Activating Token Index: 738

Click toggle to see full text

Truncated

Full Text #3


Text #4

Max Range: 0.7097. Min Range: -0.7097

Max Act: 0.6222. Min Act: -0.0000

Data Index: 1257439 (C4 (Web Text))

Max Activating Token Index: 787

Click toggle to see full text

Truncated

Full Text #4


Text #5

Max Range: 0.7097. Min Range: -0.7097

Max Act: 0.6216. Min Act: -0.0000

Data Index: 589315 (C4 (Web Text))

Max Activating Token Index: 953

Click toggle to see full text

Truncated

Full Text #5


Text #6

Max Range: 0.7097. Min Range: -0.7097

Max Act: 0.5658. Min Act: -0.0000

Data Index: 1219819 (C4 (Web Text))

Max Activating Token Index: 691

Click toggle to see full text

Truncated

Full Text #6


Text #7

Max Range: 0.7097. Min Range: -0.7097

Max Act: 0.5818. Min Act: -0.0000

Data Index: 79478 (C4 (Web Text))

Max Activating Token Index: 695

Click toggle to see full text

Truncated

Full Text #7


Text #8

Max Range: 0.7097. Min Range: -0.7097

Max Act: 0.5650. Min Act: -0.0000

Data Index: 351120 (C4 (Web Text))

Max Activating Token Index: 541

Click toggle to see full text

Truncated

Full Text #8


Text #9

Max Range: 0.7097. Min Range: -0.7097

Max Act: 0.5470. Min Act: -0.0000

Data Index: 362252 (C4 (Web Text))

Max Activating Token Index: 831

Click toggle to see full text

Truncated

Full Text #9


Text #10

Max Range: 0.7097. Min Range: -0.7097

Max Act: 0.5243. Min Act: -0.0000

Data Index: 244175 (C4 (Web Text))

Max Activating Token Index: 970

Click toggle to see full text

Truncated

Full Text #10


Text #11

Max Range: 0.7097. Min Range: -0.7097

Max Act: 0.5411. Min Act: -0.0000

Data Index: 32400 (C4 (Web Text))

Max Activating Token Index: 352

Click toggle to see full text

Truncated

Full Text #11


Text #12

Max Range: 0.7097. Min Range: -0.7097

Max Act: 0.5347. Min Act: -0.0000

Data Index: 642963 (C4 (Web Text))

Max Activating Token Index: 191

Click toggle to see full text

Truncated

Full Text #12


Text #13

Max Range: 0.7097. Min Range: -0.7097

Max Act: 0.5297. Min Act: -0.0000

Data Index: 1184745 (C4 (Web Text))

Max Activating Token Index: 575

Click toggle to see full text

Truncated

Full Text #13


Text #14

Max Range: 0.7097. Min Range: -0.7097

Max Act: 0.5040. Min Act: -0.0000

Data Index: 965882 (C4 (Web Text))

Max Activating Token Index: 197

Click toggle to see full text

Truncated

Full Text #14


Text #15

Max Range: 0.7097. Min Range: -0.7097

Max Act: 0.5048. Min Act: -0.0000

Data Index: 1215828 (C4 (Web Text))

Max Activating Token Index: 992

Click toggle to see full text

Truncated

Full Text #15


Text #16

Max Range: 0.7097. Min Range: -0.7097

Max Act: 0.5149. Min Act: -0.0000

Data Index: 1038774 (C4 (Web Text))

Max Activating Token Index: 611

Click toggle to see full text

Truncated

Full Text #16


Text #17

Max Range: 0.7097. Min Range: -0.7097

Max Act: 0.4946. Min Act: -0.0000

Data Index: 1099601 (C4 (Web Text))

Max Activating Token Index: 390

Click toggle to see full text

Truncated

Full Text #17


Text #18

Max Range: 0.7097. Min Range: -0.7097

Max Act: 0.4821. Min Act: -0.0000

Data Index: 1345428 (C4 (Web Text))

Max Activating Token Index: 850

Click toggle to see full text

Truncated

Full Text #18


Text #19

Max Range: 0.7097. Min Range: -0.7097

Max Act: 0.4673. Min Act: -0.0000

Data Index: 688206 (C4 (Web Text))

Max Activating Token Index: 735

Click toggle to see full text

Truncated

Full Text #19