Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3135 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2197. Min Range: -1.2197

Max Act: 1.2187. Min Act: -0.0000

Data Index: 949385 (The Pile)

Max Activating Token Index: 565

Text #1

Max Range: 1.2197. Min Range: -1.2197

Max Act: 1.1204. Min Act: -0.0000

Data Index: 1798842 (The Pile)

Max Activating Token Index: 436

Text #2

Max Range: 1.2197. Min Range: -1.2197

Max Act: 1.1214. Min Act: -0.0000

Data Index: 854335 (The Pile)

Max Activating Token Index: 454

Text #3

Max Range: 1.2197. Min Range: -1.2197

Max Act: 1.1823. Min Act: -0.0000

Data Index: 922158 (The Pile)

Max Activating Token Index: 81

Text #4

Max Range: 1.2197. Min Range: -1.2197

Max Act: 1.2148. Min Act: -0.0000

Data Index: 1532582 (The Pile)

Max Activating Token Index: 934

Text #5

Max Range: 1.2197. Min Range: -1.2197

Max Act: 1.1674. Min Act: -0.0000

Data Index: 1338464 (The Pile)

Max Activating Token Index: 710

Text #6

Max Range: 1.2197. Min Range: -1.2197

Max Act: 1.1461. Min Act: -0.0000

Data Index: 1100144 (The Pile)

Max Activating Token Index: 351

Text #7

Max Range: 1.2197. Min Range: -1.2197

Max Act: 1.2197. Min Act: -0.0000

Data Index: 1723673 (The Pile)

Max Activating Token Index: 526

Text #8

Max Range: 1.2197. Min Range: -1.2197

Max Act: 1.0721. Min Act: -0.0000

Data Index: 1274147 (The Pile)

Max Activating Token Index: 486

Text #9

Max Range: 1.2197. Min Range: -1.2197

Max Act: 1.1071. Min Act: -0.0000

Data Index: 686162 (The Pile)

Max Activating Token Index: 515

Text #10

Max Range: 1.2197. Min Range: -1.2197

Max Act: 1.1427. Min Act: -0.0000

Data Index: 1992588 (The Pile)

Max Activating Token Index: 754

Text #11

Max Range: 1.2197. Min Range: -1.2197

Max Act: 1.1394. Min Act: -0.0000

Data Index: 1943943 (The Pile)

Max Activating Token Index: 728

Text #12

Max Range: 1.2197. Min Range: -1.2197

Max Act: 1.0480. Min Act: -0.0000

Data Index: 755968 (The Pile)

Max Activating Token Index: 483

Text #13

Max Range: 1.2197. Min Range: -1.2197

Max Act: 1.1088. Min Act: -0.0000

Data Index: 1748726 (The Pile)

Max Activating Token Index: 283

Text #14

Max Range: 1.2197. Min Range: -1.2197

Max Act: 1.0517. Min Act: -0.0000

Data Index: 146744 (The Pile)

Max Activating Token Index: 1017

Text #15

Max Range: 1.2197. Min Range: -1.2197

Max Act: 1.0744. Min Act: -0.0000

Data Index: 1774534 (The Pile)

Max Activating Token Index: 410

Text #16

Max Range: 1.2197. Min Range: -1.2197

Max Act: 1.0613. Min Act: -0.0000

Data Index: 742109 (The Pile)

Max Activating Token Index: 347

Text #17

Max Range: 1.2197. Min Range: -1.2197

Max Act: 1.0959. Min Act: -0.0000

Data Index: 13191 (The Pile)

Max Activating Token Index: 400

Text #18

Max Range: 1.2197. Min Range: -1.2197

Max Act: 1.1273. Min Act: -0.0000

Data Index: 676187 (The Pile)

Max Activating Token Index: 399

Text #19

Max Range: 1.2197. Min Range: -1.2197

Max Act: 1.0923. Min Act: -0.0000

Data Index: 1236966 (The Pile)

Max Activating Token Index: 528