Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2574 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.7289. Min Range: -1.7289

Max Act: 1.7289. Min Act: -0.0000

Data Index: 223801 (The Pile)

Max Activating Token Index: 670

Text #1

Max Range: 1.7289. Min Range: -1.7289

Max Act: 1.1602. Min Act: -0.0000

Data Index: 1994650 (The Pile)

Max Activating Token Index: 476

Text #2

Max Range: 1.7289. Min Range: -1.7289

Max Act: 1.1101. Min Act: -0.0000

Data Index: 1452048 (The Pile)

Max Activating Token Index: 73

Text #3

Max Range: 1.7289. Min Range: -1.7289

Max Act: 1.1144. Min Act: -0.0000

Data Index: 412044 (The Pile)

Max Activating Token Index: 112

Text #4

Max Range: 1.7289. Min Range: -1.7289

Max Act: 1.0846. Min Act: -0.0000

Data Index: 1393537 (The Pile)

Max Activating Token Index: 13

Text #5

Max Range: 1.7289. Min Range: -1.7289

Max Act: 1.0795. Min Act: -0.0000

Data Index: 343624 (The Pile)

Max Activating Token Index: 440

Text #6

Max Range: 1.7289. Min Range: -1.7289

Max Act: 1.0450. Min Act: -0.0000

Data Index: 1627366 (The Pile)

Max Activating Token Index: 608

Text #7

Max Range: 1.7289. Min Range: -1.7289

Max Act: 1.0262. Min Act: -0.0000

Data Index: 755291 (The Pile)

Max Activating Token Index: 751

Text #8

Max Range: 1.7289. Min Range: -1.7289

Max Act: 1.0068. Min Act: -0.0000

Data Index: 1979474 (The Pile)

Max Activating Token Index: 355

Text #9

Max Range: 1.7289. Min Range: -1.7289

Max Act: 1.0036. Min Act: -0.0000

Data Index: 991887 (The Pile)

Max Activating Token Index: 679

Text #10

Max Range: 1.7289. Min Range: -1.7289

Max Act: 0.9576. Min Act: -0.0000

Data Index: 35297 (The Pile)

Max Activating Token Index: 340

Text #11

Max Range: 1.7289. Min Range: -1.7289

Max Act: 0.9819. Min Act: -0.0000

Data Index: 1559531 (The Pile)

Max Activating Token Index: 409

Text #12

Max Range: 1.7289. Min Range: -1.7289

Max Act: 0.9861. Min Act: -0.0000

Data Index: 1775094 (The Pile)

Max Activating Token Index: 288

Text #13

Max Range: 1.7289. Min Range: -1.7289

Max Act: 0.9569. Min Act: -0.0000

Data Index: 1325097 (The Pile)

Max Activating Token Index: 55

Text #14

Max Range: 1.7289. Min Range: -1.7289

Max Act: 0.9755. Min Act: -0.0000

Data Index: 514438 (The Pile)

Max Activating Token Index: 991

Text #15

Max Range: 1.7289. Min Range: -1.7289

Max Act: 1.0061. Min Act: -0.0000

Data Index: 984592 (The Pile)

Max Activating Token Index: 174

Text #16

Max Range: 1.7289. Min Range: -1.7289

Max Act: 0.9587. Min Act: -0.0000

Data Index: 1016150 (The Pile)

Max Activating Token Index: 642

Text #17

Max Range: 1.7289. Min Range: -1.7289

Max Act: 0.9562. Min Act: -0.0000

Data Index: 1488010 (The Pile)

Max Activating Token Index: 24

Text #18

Max Range: 1.7289. Min Range: -1.7289

Max Act: 0.9267. Min Act: -0.0000

Data Index: 135219 (The Pile)

Max Activating Token Index: 221

Text #19

Max Range: 1.7289. Min Range: -1.7289

Max Act: 0.9452. Min Act: -0.0000

Data Index: 732557 (The Pile)

Max Activating Token Index: 532