Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4550 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.8261. Min Range: -1.8261

Max Act: 1.7819. Min Act: -0.0000

Data Index: 988342 (The Pile)

Max Activating Token Index: 334

Text #1

Max Range: 1.8261. Min Range: -1.8261

Max Act: 1.5490. Min Act: -0.0000

Data Index: 513781 (The Pile)

Max Activating Token Index: 723

Text #2

Max Range: 1.8261. Min Range: -1.8261

Max Act: 1.8261. Min Act: -0.0000

Data Index: 1601648 (The Pile)

Max Activating Token Index: 249

Text #3

Max Range: 1.8261. Min Range: -1.8261

Max Act: 1.6954. Min Act: -0.0000

Data Index: 1397035 (The Pile)

Max Activating Token Index: 314

Text #4

Max Range: 1.8261. Min Range: -1.8261

Max Act: 1.7024. Min Act: -0.0000

Data Index: 953615 (The Pile)

Max Activating Token Index: 253

Text #5

Max Range: 1.8261. Min Range: -1.8261

Max Act: 1.6717. Min Act: -0.0000

Data Index: 227388 (The Pile)

Max Activating Token Index: 500

Text #6

Max Range: 1.8261. Min Range: -1.8261

Max Act: 1.5475. Min Act: -0.0000

Data Index: 1838168 (The Pile)

Max Activating Token Index: 503

Text #7

Max Range: 1.8261. Min Range: -1.8261

Max Act: 1.5959. Min Act: -0.0000

Data Index: 1771810 (The Pile)

Max Activating Token Index: 569

Text #8

Max Range: 1.8261. Min Range: -1.8261

Max Act: 1.6249. Min Act: -0.0000

Data Index: 738632 (The Pile)

Max Activating Token Index: 555

Text #9

Max Range: 1.8261. Min Range: -1.8261

Max Act: 1.7080. Min Act: -0.0000

Data Index: 400026 (The Pile)

Max Activating Token Index: 647

Text #10

Max Range: 1.8261. Min Range: -1.8261

Max Act: 1.5180. Min Act: -0.0000

Data Index: 1247567 (The Pile)

Max Activating Token Index: 301

Text #11

Max Range: 1.8261. Min Range: -1.8261

Max Act: 1.6101. Min Act: -0.0000

Data Index: 289459 (The Pile)

Max Activating Token Index: 168

Text #12

Max Range: 1.8261. Min Range: -1.8261

Max Act: 1.4698. Min Act: -0.0000

Data Index: 148182 (The Pile)

Max Activating Token Index: 642

Text #13

Max Range: 1.8261. Min Range: -1.8261

Max Act: 1.2478. Min Act: -0.0000

Data Index: 569473 (The Pile)

Max Activating Token Index: 823

Text #14

Max Range: 1.8261. Min Range: -1.8261

Max Act: 1.4071. Min Act: -0.0000

Data Index: 1757600 (The Pile)

Max Activating Token Index: 555

Text #15

Max Range: 1.8261. Min Range: -1.8261

Max Act: 1.4841. Min Act: -0.0000

Data Index: 554574 (The Pile)

Max Activating Token Index: 639

Text #16

Max Range: 1.8261. Min Range: -1.8261

Max Act: 1.6620. Min Act: -0.0000

Data Index: 1954288 (The Pile)

Max Activating Token Index: 303

Text #17

Max Range: 1.8261. Min Range: -1.8261

Max Act: 1.5742. Min Act: -0.0000

Data Index: 950858 (The Pile)

Max Activating Token Index: 423

Text #18

Max Range: 1.8261. Min Range: -1.8261

Max Act: 1.5372. Min Act: -0.0000

Data Index: 666941 (The Pile)

Max Activating Token Index: 529

Text #19

Max Range: 1.8261. Min Range: -1.8261

Max Act: 1.5075. Min Act: -0.0000

Data Index: 1492466 (The Pile)

Max Activating Token Index: 443