Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2943 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.3803. Min Range: -5.3803

Max Act: 5.3803. Min Act: -0.0000

Data Index: 43203 (The Pile)

Max Activating Token Index: 834

Text #1

Max Range: 5.3803. Min Range: -5.3803

Max Act: 4.6457. Min Act: -0.0000

Data Index: 453622 (The Pile)

Max Activating Token Index: 905

Text #2

Max Range: 5.3803. Min Range: -5.3803

Max Act: 4.6922. Min Act: -0.0000

Data Index: 1547691 (The Pile)

Max Activating Token Index: 249

Text #3

Max Range: 5.3803. Min Range: -5.3803

Max Act: 4.5617. Min Act: -0.0000

Data Index: 864615 (The Pile)

Max Activating Token Index: 490

Text #4

Max Range: 5.3803. Min Range: -5.3803

Max Act: 5.0249. Min Act: -0.0000

Data Index: 1395446 (The Pile)

Max Activating Token Index: 602

Text #5

Max Range: 5.3803. Min Range: -5.3803

Max Act: 4.2562. Min Act: -0.0000

Data Index: 1713195 (The Pile)

Max Activating Token Index: 741

Text #6

Max Range: 5.3803. Min Range: -5.3803

Max Act: 4.4803. Min Act: -0.0000

Data Index: 1565025 (The Pile)

Max Activating Token Index: 934

Text #7

Max Range: 5.3803. Min Range: -5.3803

Max Act: 4.5842. Min Act: -0.0000

Data Index: 1798189 (The Pile)

Max Activating Token Index: 697

Text #8

Max Range: 5.3803. Min Range: -5.3803

Max Act: 4.4622. Min Act: -0.0000

Data Index: 1825626 (The Pile)

Max Activating Token Index: 856

Text #9

Max Range: 5.3803. Min Range: -5.3803

Max Act: 4.4707. Min Act: -0.0000

Data Index: 1469374 (The Pile)

Max Activating Token Index: 534

Text #10

Max Range: 5.3803. Min Range: -5.3803

Max Act: 4.4470. Min Act: -0.0000

Data Index: 434345 (The Pile)

Max Activating Token Index: 263

Text #11

Max Range: 5.3803. Min Range: -5.3803

Max Act: 4.4072. Min Act: -0.0000

Data Index: 401632 (The Pile)

Max Activating Token Index: 956

Text #12

Max Range: 5.3803. Min Range: -5.3803

Max Act: 4.6341. Min Act: -0.0000

Data Index: 1844521 (The Pile)

Max Activating Token Index: 905

Text #13

Max Range: 5.3803. Min Range: -5.3803

Max Act: 4.5034. Min Act: -0.0000

Data Index: 466270 (The Pile)

Max Activating Token Index: 693

Text #14

Max Range: 5.3803. Min Range: -5.3803

Max Act: 4.3543. Min Act: -0.0000

Data Index: 1089655 (The Pile)

Max Activating Token Index: 465

Text #15

Max Range: 5.3803. Min Range: -5.3803

Max Act: 3.6884. Min Act: -0.0000

Data Index: 1383789 (The Pile)

Max Activating Token Index: 702

Text #16

Max Range: 5.3803. Min Range: -5.3803

Max Act: 3.9166. Min Act: -0.0000

Data Index: 1789482 (The Pile)

Max Activating Token Index: 358

Text #17

Max Range: 5.3803. Min Range: -5.3803

Max Act: 3.8392. Min Act: -0.0000

Data Index: 1567416 (The Pile)

Max Activating Token Index: 518

Text #18

Max Range: 5.3803. Min Range: -5.3803

Max Act: 3.9459. Min Act: -0.0000

Data Index: 1768688 (The Pile)

Max Activating Token Index: 166

Text #19

Max Range: 5.3803. Min Range: -5.3803

Max Act: 3.8278. Min Act: -0.0000

Data Index: 306887 (The Pile)

Max Activating Token Index: 130