< Prev | Home | Model | Random | Next >

Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 5811 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('solu-12l')

Click the toggle to see the full text!



Text #0

Max Range: 1.4980. Min Range: -1.4980

Max Act: 1.4980. Min Act: -0.0002

Data Index: 778283 (C4 (Web Text))

Max Activating Token Index: 710

Click toggle to see full text

Truncated

Full Text #0


Text #1

Max Range: 1.4980. Min Range: -1.4980

Max Act: 1.3713. Min Act: -0.0002

Data Index: 979452 (C4 (Web Text))

Max Activating Token Index: 10

Click toggle to see full text

Truncated

Full Text #1


Text #2

Max Range: 1.4980. Min Range: -1.4980

Max Act: 1.2964. Min Act: -0.0002

Data Index: 1121388 (C4 (Web Text))

Max Activating Token Index: 716

Click toggle to see full text

Truncated

Full Text #2


Text #3

Max Range: 1.4980. Min Range: -1.4980

Max Act: 1.2606. Min Act: -0.0002

Data Index: 899604 (C4 (Web Text))

Max Activating Token Index: 560

Click toggle to see full text

Truncated

Full Text #3


Text #4

Max Range: 1.4980. Min Range: -1.4980

Max Act: 1.1930. Min Act: -0.0002

Data Index: 449752 (C4 (Web Text))

Max Activating Token Index: 778

Click toggle to see full text

Truncated

Full Text #4


Text #5

Max Range: 1.4980. Min Range: -1.4980

Max Act: 1.2040. Min Act: -0.0002

Data Index: 716985 (C4 (Web Text))

Max Activating Token Index: 66

Click toggle to see full text

Truncated

Full Text #5


Text #6

Max Range: 1.4980. Min Range: -1.4980

Max Act: 1.1618. Min Act: -0.0002

Data Index: 1293314 (C4 (Web Text))

Max Activating Token Index: 870

Click toggle to see full text

Truncated

Full Text #6


Text #7

Max Range: 1.4980. Min Range: -1.4980

Max Act: 1.1396. Min Act: -0.0002

Data Index: 382305 (C4 (Web Text))

Max Activating Token Index: 280

Click toggle to see full text

Truncated

Full Text #7


Text #8

Max Range: 1.4980. Min Range: -1.4980

Max Act: 1.1088. Min Act: -0.0002

Data Index: 99006 (C4 (Web Text))

Max Activating Token Index: 329

Click toggle to see full text

Truncated

Full Text #8


Text #9

Max Range: 1.4980. Min Range: -1.4980

Max Act: 1.1102. Min Act: -0.0002

Data Index: 1258858 (C4 (Web Text))

Max Activating Token Index: 107

Click toggle to see full text

Truncated

Full Text #9


Text #10

Max Range: 1.4980. Min Range: -1.4980

Max Act: 1.1109. Min Act: -0.0002

Data Index: 819470 (C4 (Web Text))

Max Activating Token Index: 309

Click toggle to see full text

Truncated

Full Text #10


Text #11

Max Range: 1.4980. Min Range: -1.4980

Max Act: 1.0822. Min Act: -0.0002

Data Index: 1340354 (C4 (Web Text))

Max Activating Token Index: 470

Click toggle to see full text

Truncated

Full Text #11


Text #12

Max Range: 1.4980. Min Range: -1.4980

Max Act: 1.0919. Min Act: -0.0002

Data Index: 777514 (C4 (Web Text))

Max Activating Token Index: 687

Click toggle to see full text

Truncated

Full Text #12


Text #13

Max Range: 1.4980. Min Range: -1.4980

Max Act: 1.0892. Min Act: -0.0002

Data Index: 979219 (C4 (Web Text))

Max Activating Token Index: 672

Click toggle to see full text

Truncated

Full Text #13


Text #14

Max Range: 1.4980. Min Range: -1.4980

Max Act: 1.0684. Min Act: -0.0002

Data Index: 1140569 (C4 (Web Text))

Max Activating Token Index: 86

Click toggle to see full text

Truncated

Full Text #14


Text #15

Max Range: 1.4980. Min Range: -1.4980

Max Act: 1.0653. Min Act: -0.0002

Data Index: 989736 (C4 (Web Text))

Max Activating Token Index: 828

Click toggle to see full text

Truncated

Full Text #15


Text #16

Max Range: 1.4980. Min Range: -1.4980

Max Act: 1.0723. Min Act: -0.0002

Data Index: 451134 (C4 (Web Text))

Max Activating Token Index: 489

Click toggle to see full text

Truncated

Full Text #16


Text #17

Max Range: 1.4980. Min Range: -1.4980

Max Act: 1.0416. Min Act: -0.0002

Data Index: 518531 (C4 (Web Text))

Max Activating Token Index: 175

Click toggle to see full text

Truncated

Full Text #17


Text #18

Max Range: 1.4980. Min Range: -1.4980

Max Act: 1.0674. Min Act: -0.0002

Data Index: 906457 (C4 (Web Text))

Max Activating Token Index: 722

Click toggle to see full text

Truncated

Full Text #18


Text #19

Max Range: 1.4980. Min Range: -1.4980

Max Act: 1.0584. Min Act: -0.0002

Data Index: 930050 (C4 (Web Text))

Max Activating Token Index: 677

Click toggle to see full text

Truncated

Full Text #19