Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 399 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.5029. Min Range: -1.5029

Max Act: 1.5029. Min Act: -0.0000

Data Index: 78271 (The Pile)

Max Activating Token Index: 917

Text #1

Max Range: 1.5029. Min Range: -1.5029

Max Act: 1.3748. Min Act: -0.0000

Data Index: 632601 (The Pile)

Max Activating Token Index: 22

Text #2

Max Range: 1.5029. Min Range: -1.5029

Max Act: 1.2160. Min Act: -0.0000

Data Index: 1647698 (The Pile)

Max Activating Token Index: 226

Text #3

Max Range: 1.5029. Min Range: -1.5029

Max Act: 1.1516. Min Act: -0.0000

Data Index: 587652 (The Pile)

Max Activating Token Index: 199

Text #4

Max Range: 1.5029. Min Range: -1.5029

Max Act: 1.1256. Min Act: -0.0000

Data Index: 1416027 (The Pile)

Max Activating Token Index: 104

Text #5

Max Range: 1.5029. Min Range: -1.5029

Max Act: 1.1048. Min Act: -0.0000

Data Index: 227054 (The Pile)

Max Activating Token Index: 752

Text #6

Max Range: 1.5029. Min Range: -1.5029

Max Act: 0.8963. Min Act: -0.0000

Data Index: 523904 (The Pile)

Max Activating Token Index: 818

Text #7

Max Range: 1.5029. Min Range: -1.5029

Max Act: 1.0430. Min Act: -0.0000

Data Index: 1920573 (The Pile)

Max Activating Token Index: 276

Text #8

Max Range: 1.5029. Min Range: -1.5029

Max Act: 0.9992. Min Act: -0.0000

Data Index: 1838861 (The Pile)

Max Activating Token Index: 651

Text #9

Max Range: 1.5029. Min Range: -1.5029

Max Act: 0.9336. Min Act: -0.0000

Data Index: 939337 (The Pile)

Max Activating Token Index: 735

Text #10

Max Range: 1.5029. Min Range: -1.5029

Max Act: 0.9561. Min Act: -0.0000

Data Index: 342344 (The Pile)

Max Activating Token Index: 364

Text #11

Max Range: 1.5029. Min Range: -1.5029

Max Act: 0.8040. Min Act: -0.0000

Data Index: 871730 (The Pile)

Max Activating Token Index: 933

Text #12

Max Range: 1.5029. Min Range: -1.5029

Max Act: 0.8894. Min Act: -0.0000

Data Index: 1778290 (The Pile)

Max Activating Token Index: 556

Text #13

Max Range: 1.5029. Min Range: -1.5029

Max Act: 0.9034. Min Act: -0.0000

Data Index: 414596 (The Pile)

Max Activating Token Index: 457

Text #14

Max Range: 1.5029. Min Range: -1.5029

Max Act: 0.8846. Min Act: -0.0000

Data Index: 1134810 (The Pile)

Max Activating Token Index: 435

Text #15

Max Range: 1.5029. Min Range: -1.5029

Max Act: 0.8375. Min Act: -0.0000

Data Index: 1465210 (The Pile)

Max Activating Token Index: 464

Text #16

Max Range: 1.5029. Min Range: -1.5029

Max Act: 0.9485. Min Act: -0.0000

Data Index: 159457 (The Pile)

Max Activating Token Index: 150

Text #17

Max Range: 1.5029. Min Range: -1.5029

Max Act: 0.8300. Min Act: 0.0000

Data Index: 846888 (The Pile)

Max Activating Token Index: 726

Text #18

Max Range: 1.5029. Min Range: -1.5029

Max Act: 0.8339. Min Act: -0.0000

Data Index: 462662 (The Pile)

Max Activating Token Index: 41

Text #19

Max Range: 1.5029. Min Range: -1.5029

Max Act: 0.8065. Min Act: -0.0000

Data Index: 663264 (The Pile)

Max Activating Token Index: 865